What is the tty in custom command

anon50555658 · June 9, 2024, 2:45am

Hello!
I am probably not really qualified to post in this part of the forum, so if I am overstepping, please move the post to the appropriate place.

I maintain a bash script called shrink-backup that makes small img backups, primarily aimed at single board computers.

I got a bug report today involving webmin and custom command that I can’t seem to figure out so I turn directly to the experts.

Short description:
I use dd to copy the boot partition basically, and to get the output on console I pipe it into /dev/tty with tee.

Problems arise when there IS no tty.
Not sure why I can not validate this with [ -z /dev/tty ] though, but I would prefer to know the where the console is so I can give the same functionality if the script is ran as a custom command in webmin instead of repiping it to /dev/null anyway.

I ran tty and put the output into a logfile and it nicely returned “not a tty”. xD
who am i returned nothing.

Any ideas?

Thanks in advance.

Joe · June 9, 2024, 3:24am

A Custom Command in Webmin isn’t an interactive terminal. There is no TTY.

Webmin has an ineractive terminal (called “Terminal”), but you can’t use it like Custom Commands. Custom Commands just runs a command…it cannot be interactive.

I’m not sure I understand why you’re piping to /dev/tty instead of just letting it go to STDOUT (which is where the vast majority of standard CLI tools normally goes, right?)? STDOUT will show up on the output page of the Custom Command.

anon50555658 · June 9, 2024, 3:31am

IIRC when I made that ages ago, the reason is that I also have to gather it into a file.

So IIRC STDOUT got suppressed somehow (I am talking about the counter while dd is running if you use option staus=progress here)

The full line is if ! output=$(dd bs=512 count=$LOCAL_DDBOOTSECTOR if="$LOCAL_DEV_PATH" of="$IMG_FILE" conv=noerror,sync status=progress 2>&1 | tee /dev/tty); then

I tried to do it differently but this was the only way I could get it to work.

Joe · June 9, 2024, 3:34am

I suspect status=progress will also cause a problem for a noninteractive terminal. I don’t remember how it handles output, but if it’s moving the cursor, Custom Common won’t do anything good with it (it is also not capable of escape sequences for cursor control…it is not at all a terminal).

anon50555658 · June 9, 2024, 3:37am

Well, it works on ssh and all other interaction methods I have tried, only this that has failed with this so far…
It also works If I run the command in the command prompt I can open in the bottom left in the webmin gui.
It’s only in the custom command it fails.

I was hoping there was something simple I could check and just pipe it to that instead.

Joe · June 9, 2024, 3:39am

I mean, it is, in fact, simple. It’s simply not a terminal.

Your simple solution is to don’t do anything that requires a terminal. No /dev/tty, send output to STDOUT. No cursor actions, no colors (though that’ll just look messy, it won’t prevent it from working, I don’t think), etc. If it works in a cronjob, it’ll probably work in Custom Commands.

anon50555658 · June 9, 2024, 3:42am

Yes, I was trying to achieve that by running [ -z /dev/tty ] but for some reason that returns as false even though running an ls and catching the output, there IS NO /dev/tty.

And completely removing the functionality for ALL others is out of the question.

If I can just figure out a way to find out if /dev/tty is available or not somehow, I fix it by just routing it to /dev/null …
And the webmin users will have to do without the counter and a few other things usually reported to the user via terminal.

Edit
My idea was this:

- if ! output=$(dd bs=512 count=$LOCAL_DDBOOTSECTOR if="$LOCAL_DEV_PATH" of="$IMG_FILE" conv=noerror,sync status=progress 2>&1 | tee /dev/tty); then
+ if [ -z /dev/tty ]; then
+   TTY_AVAILABILITY='/dev/null'
+ else
+   TTY_AVAILABILITY='/dev/tty'
+ fi
+ if ! output=$(dd bs=512 count=$LOCAL_DDBOOTSECTOR if="$LOCAL_DEV_PATH" of="$IMG_FILE" conv=noerror,sync status=progress 2>&1 | tee "${TTY_AVAILABILITY}"); then

Joe · June 9, 2024, 3:57am

Seems like you could do if tty -s instead? You said tty knows it’s not a terminal.

Joe · June 9, 2024, 3:58am

And Webmin users can run it in an interactive terminal by using the Terminal (but, that’s not a pre-configured button).

anon50555658 · June 9, 2024, 4:12am

yeah, I got it working for checking exactly that string. xD
Edit Or wait, -s, I somehow missed to read the help on tty, thanks for pointing that out.
Edit 2 That actually ALSO does not work, maybe for the same reason the -z with the test returned incorrectly, but I’ll just catch the string ‘not a tty’ and call it a win unless you have further input about below end edit 2

My brain was not working correctly.
So you can’t recommed an output I can pipe to instead of /dev/null?
Because the script works with that, but the counter in dd (among other stuff in the script) will not be communicated other than if the user chooses to utilize a logfile when using a custom command.

Joe · June 9, 2024, 5:24am

Why doesn’t STDOUT work? (i.e. don’t pipe it anywhere)

I’ve never been clear on that. It seems like something weird is going on if you can’t use STDOUT. It’s how everything works.

I’m able to run dd with status=progress (this doesn’t give any output until it’s finished running, which I think is expected). This is in the Command Shell (which is also a non-interactive terminal, just one you can run commands without creating it…just type it in and hit Execute).

So, that would work fine in Custom Commands.

anon50555658 · June 9, 2024, 11:47am

Try running the command manually in a terminal. It is NOT supposed to be a report after, it is supposed to print the copy live just as the option suggests “status=PROGRESS”. It’s NOT supposed to print mulitple lines, only the one that counts up until the operation is done.
Why it works in cli but not in script (or in webmin things), I have no idea, kinda why I came here to ask… xD

The “execute command” just spins forever on my installation so I never bothered with it.
I figured I did something wrong in the installation or smthn, but I also only installed to try to help the user that made the bug report so I wont spend time on it.

The webmin users will have to do with a sub par version if running with “custom command”.
Unless you can recommend another location I can pipe it to, we can consider this solved.

A bit sad, I was hoping to be able to deliver the same quality on all platforms, but it is what it is…

Thank you so much!

Ilia · June 9, 2024, 12:59pm

Thanks for the heads up! Yes, you’re right – this is a bug!

We will fix it in the next Webmin and Authentic theme release.

anon50555658 · June 9, 2024, 1:13pm

Wow!

~~Could I be annoying and ask, fix how?~~

~~What I mean is, will there be some kind of dev I can pipe to?~~
~~Or you mean that the functionality of status=progress will get its output fixed?~~

~~I’m asking because I want to know what or even if I have to change something in my script.~~

Actually, nvm, I’ll just implement a check anyway, and remove the output for webmin users, because I have a suspicion my problem wont get solved by that anyway.

Ilia · June 9, 2024, 1:23pm

If you are a developer, you could install the latest nightly version of Webmin, which includes the latest fix. However, I have been distracted by numerous issues and haven’t implemented the client-side for the new live status collection after we switched the backend to WebSockets. So, if you’re ready to temporarily disable Real-time monitoring on the Theme Configuration: Dashboard and real-time monitoring page, you can go ahead and grab the latest nightly version of Webmin to get the just-implemented fix right now.

anon50555658 · June 9, 2024, 1:47pm

I mean… I DO maintain a script… xD

I uninstalled using the uninstall script, downloaded you script for adding the keys for the devel, installed that and ran the script.

As I suspected, it did not change anything for me.

Output from /home/bedna/bin/shrink-backup -atlyz /mnt/backup/rpi4b8gb/piTest/piTest.img ..

## Debugging requested, writing to log file /home/bedna/bin/shrink-backup.log
## Zoom speed requested...
## Scanning filesystem and calculating...
##############################################################################
# DISABLE PROMPTS SELECTED (-y), NO WARNINGS ABOUT DELETION!!!
# A backup will be created at /mnt/backup/rpi4b8gb/piTest/piTest.img
# ext4 filesystem detected on root
# ----------------------------------------------------------------------------
# Write to logfile: true
# Zoom speed requested: true
# Autocalculate img root partition size: true
# Autoexpand filesystem at boot: true
# Use exclude.txt: true
# Bootsector size: 512MiB
# Estemated root usage: 2456MiB
# Auto calculated size (root partition): 3240MiB
# Total img size: 3752MiB
!! Removing old img file...
## Creating bootsector...
tee: /dev/tty: No such device or address
e[0;33m
19800576 bytes (20 MB, 19 MiB) copied, 1 s, 19.7 MB/s
38937088 bytes (39 MB, 37 MiB) copied, 2 s, 19.4 MB/s
58335744 bytes (58 MB, 56 MiB) copied, 3 s, 19.4 MB/s
77341184 bytes (77 MB, 74 MiB) copied, 4 s, 19.3 MB/s
96084480 bytes (96 MB, 92 MiB) copied, 5 s, 19.2 MB/s
115220992 bytes (115 MB, 110 MiB) copied, 6 s, 19.2 MB/s
134488576 bytes (134 MB, 128 MiB) copied, 7 s, 19.2 MB/s
153625088 bytes (154 MB, 147 MiB) copied, 8 s, 19.2 MB/s
172892672 bytes (173 MB, 165 MiB) copied, 9 s, 19.2 MB/s
192291328 bytes (192 MB, 183 MiB) copied, 10 s, 19.2 MB/s
210772480 bytes (211 MB, 201 MiB) copied, 11 s, 19.2 MB/s
230040064 bytes (230 MB, 219 MiB) copied, 12 s, 19.2 MB/s
249176576 bytes (249 MB, 238 MiB) copied, 13 s, 19.2 MB/s
268444160 bytes (268 MB, 256 MiB) copied, 14 s, 19.2 MB/s
287580672 bytes (288 MB, 274 MiB) copied, 15 s, 19.2 MB/s
306323968 bytes (306 MB, 292 MiB) copied, 16 s, 19.1 MB/s
325460480 bytes (325 MB, 310 MiB) copied, 17 s, 19.1 MB/s
344859136 bytes (345 MB, 329 MiB) copied, 18 s, 19.2 MB/s
364151808 bytes (364 MB, 347 MiB) copied, 19 s, 19.2 MB/s
383263232 bytes (383 MB, 366 MiB) copied, 20 s, 19.2 MB/s
402006528 bytes (402 MB, 383 MiB) copied, 21 s, 19.1 MB/s
421143040 bytes (421 MB, 402 MiB) copied, 22 s, 19.1 MB/s
440516096 bytes (441 MB, 420 MiB) copied, 23 s, 19.2 MB/s
459547136 bytes (460 MB, 438 MiB) copied, 24 s, 19.1 MB/s
478945792 bytes (479 MB, 457 MiB) copied, 25 s, 19.2 MB/s
497991168 bytes (498 MB, 475 MiB) copied, 26 s, 19.2 MB/s
516825600 bytes (517 MB, 493 MiB) copied, 27 s, 19.1 MB/s
536093184 bytes (536 MB, 511 MiB) copied, 28 s, 19.1 MB/s
1069056+0 records in
1069056+0 records out
547356672 bytes (547 MB, 522 MiB) copied, 31.2465 s, 17.5 MB/s
e[0;31m!! DD TO LOCAL_BOOTSECTOR FAILED!!!e[0m
## Cleanup function called with non zero exit code, something went wrong!!!
## Exiting and cleaning up...
## Please stand by...
## Done.
## Elapsed time: 00.33

Command failed with exit status 256

Ilia · June 9, 2024, 1:53pm

Do you now see the progressive output in the Command Shell module when running a command after the theme update? You can try with a simple command like ping -c 10 google.com.

Remember to force reload the page after Webmin upgrade.

Ilia · June 9, 2024, 2:23pm

Check this simple PR. It will fix the issue for you!

anon50555658 · June 9, 2024, 2:36pm

Oh, you meant that, yes, that works now!

And I got the script working too, but the problem is the userfeedback is almost completely missing for big parts of the script, but at least it works.
I’ll push it to testing branch soon, I just need to clean up some code. I am a noob and dont push directly from my editor, I manually change the files on github.

Edit

Output from /home/bedna/bin/shrink-backup -atlyz /mnt/backup/rpi4b8gb/piTest/piTest.img ..

## Debugging requested, writing to log file /home/bedna/bin/shrink-backup.log
## Zoom speed requested...
## Scanning filesystem and calculating...
##############################################################################
# DISABLE PROMPTS SELECTED (-y), NO WARNINGS ABOUT DELETION!!!
# A backup will be created at /mnt/backup/rpi4b8gb/piTest/piTest.img
# ext4 filesystem detected on root
# ----------------------------------------------------------------------------
# Write to logfile: true
# Zoom speed requested: true
# Autocalculate img root partition size: true
# Autoexpand filesystem at boot: true
# Use exclude.txt: true
# Bootsector size: 512MiB
# Estemated root usage: 2457MiB
# Auto calculated size (root partition): 2989MiB
# Total img size: 3501MiB
!! Removing old img file...
## Creating bootsector...
## Resizing img file...
## Looping img file...
## Removing root partition...
## Recreating root partition...
## Formatting filesystem...
## Checking img filesystem...
rootfs: 11/191232 files (0.0% non-contiguous), 30640/764335 blocks
## Creating temp directory...
## Mounting img root partition...
## Mounting img boot partition...
## Backing up files...
## Rsync done...
## Please stand by...
## Finalizing filesystem...
rootfs: /lost+found not found.  CREATED.

       96373 inodes used (50.40%, out of 191232)
          14 non-contiguous files (0.0%)
          58 non-contiguous directories (0.1%)
             # of inodes with ind/dind/tind blocks: 0/0/0
             Extent depth histogram: 90395/1
      608361 blocks used (79.59%, out of 764335)
           0 bad blocks
           1 large file

       82794 regular files
        7500 directories
           0 character device files
           0 block device files
           0 fifos
         366 links
        6069 symbolic links (5968 fast symbolic links)
           0 sockets
------------
       96729 files
## Remounting for autoexpansion...
## Mounting img root partition...
## Mounting img boot partition...
## Enabling fs-autoexpand...
## Raspberry pi detected...
## Raspberry pi filesystem autoresizing at boot...
## Backup done.
##############################################################################
## Write to logfile: true
## Autoexpand filesystem at boot: true
## Use exclude.txt: true
## /mnt/backup/rpi4b8gb/piTest/piTest.img is 3501MiB with a root partition of 2989MiB.
## Please wait for the system to reboot after restoring an image with autoexpansion.
##############################################################################
## Exiting and cleaning up...
## Please stand by...
## Done.
## Elapsed time: 01.59

Edit 2
There, it is now working on the testing branch.

Thank you so much!

system · October 7, 2024, 2:37pm

This topic was automatically closed 120 days after the last reply. New replies are no longer allowed.