That is all I can get from the dashboard.
This is Rocky Linux 8.10
Plenty of ram and disk space.
Runs Apache, Postfix, and is one of my slave nameservers.
This is a HP Proliant DL380 on Raid.
All updates were installed several days ago. There are some new now that haven’t been installed.
This is our internal system. It is located in the office without battery backup unlike the systems in our server building.
When I go to virtualmin, at first I can see Cloudmin, Virtualmin and Webmin tabs at the top of the left pane. When I go to Cloudmin or Virtualmin, it says the module is not installed. After that, the tabs at the top of that pane change to Webmin and Dashboard. Weird.
Looking through Webmin, a lot of features are missing such as Bootup and Shutdown, Apache doesn’t show under servers, I’m sure there are more. Software Updates is also missing.
I do have backups of /etc. Apache, postfix, mariadb and dovecot are functioning. It seems only the Webmin trio is broken.
First:
Can someone provide which file(s) seem to be corrupted? I did look at the file sizes of the base config file against my backups and they have not changed although the file dates have changed. I’m not sure if Cloudmin stores more data somewhere else. Should I try restoring a backup of the config file? Or is there a known solution for this?
Second:
I have never had this type of thing happen on any OS or with any software installed on the OS. I have lost edits on open files of course. How can this be happening? This is not the first time for me. I don’t get it. Shouldn’t config change be written to disk immediately?
It’s been a long time since the ‘bad old days’ of ext2. It was always so much fun to reboot after a power failure. In theory the ext3 system doesn’t erase the original until the new is complete. Supposedly, IF I remember correctly, the last thing it does is set the flag on the old to mark the space unused.
Well, I guess the good news is it booted? Did you happen to watch the boot sequence for orphaned inodes? I’d suggest booting a rescue disk such as gparted live and check the file system as a first step.
But battery backup and SCSI drives are the only real protection. SCSI drives have batteries that are just enough to ensure all disk transactions complete.
Are the modules actually gone or are they just disabled for the user you are logging in as?
That would be in the acl files and miniserv.users in /etc/webmin. So, check those against your backups.
Then check to see if the modules are still there in /usr/libexec/webmin. e.g. look for the apache directory and see if it has the expected pile of files in it.
Before you start trying to solve the problem, I strongly recommend you identify the problem with specificity and understand it. A lot of folks start randomly uninstalling and reinstalling stuff, etc. without even knowing what was wrong to start with, and that almost always just digs the hole deeper.
It is not starting up on reboot. That would be the root user. When I log in as root, I quite simply don’t see Cloudmin nor Virtualmin tabs within the interface.
I did a diff on the config file. I do use rsnapshot for backups and 6 day, 3 weeks and 2 months, so I have 11 to look at. The recent backup config file only had a datestamp change for letsencrypt. Older ones has some more shanges which all made sense.
I did start Apache manually. All other services that I know of did start. Apache, SSH, Dovecot, Postfix, Named. I did have to start fail2ban manually. I can’t find anything that does not function beyond some modules in Webmin not showing, nothing in Cloudmin nor Virtualmin is available through the interface.
In Webmin, the Apache module shows and at least can start and stop apache. Same for dovecot and postfix. Bind does not show. Running processes, bootup and shutdown and others are not shown.
I have restarted Webmin a few times. That doesn’t seem to make any difference. I’m not sure if Webmin should be starting Virtualmin and Cloudmin or if those have their own startup commands?
I have looked through /user/libexec/webmin. I find the apache directory, loaded with tons of files. virtual-server also exist. Tons of files. cloudmin-services also exist, with far fewer files. I have no way of comparing this as I do not have a backup of that directory.
Can you help with which particular modules to look for? What module is being looked for by the interface and reported as missing?
This is a live and important system. It was booted twice on Sunday due to power outages. I really need to not do this except in the wee hours of the morning at this point.
Look at the specific files I told you to look at in /etc/webmin to see if they have changed.
I don’t believe the modules are gone, based on all of the evidence you’ve found so far. I believe your user(s) simply don’t have access to them, which is a function of the files I mentioned. If you have backups of those files (the .acl files and the miniserv.users file) and you can compare them, you will likely see what changed that made those modules unavailable.
You’ve confirmed the modules are still there (“tons of files” in /usr/libexec/webmin), so focus on the users and the files I mentioned.
This is introducing a new issue and should be taken up in another topic:
There’s already a lot going on here. Let’s focus on one problem at a time. If it starts when you tell it to (as I assume it must, since you’re able to login to see modules missing from the GUI), then the service is probably just disabled. If you need to ask followup questions about it not starting on boot, make a new topic.
I also compared /etc/webmin/miniserve.users and /etc/webmin/acl/config to those files on another working system. The config files are identical. The miniserve.users files are much the same. Only the usernames do not match. oneofmydomains.com:::::::0:0::: is an example from one of these.
I am logged in as root which is identical to other working systems in the miniserv.users file.
I have made no changes to this system since the outage. I have only started the system after the outage and done one shutdown and reboot afterwards when getting off the generator.
I believe that addresses all of your suggestions. If not, please let me know.
I’m not sure where to find the version numbers for Virtualmin and Cloudmin. Virtualmin should be on 7.30.4 Pro. I have no idea about Cloudmin other than it is the paid version. I had run updates within the week or so before the power outage. No updates have been run since.
All modules shown have checks. None have no check marks.
Of note, the modules I cannot see in the Webmin interface don’t seem to show in this area. For instance, Startup and Shutdown does not appear in either the interface or on the list.
Unused Modules is one of the modules that does not display in the interface.
(But, don’t do dnf remove webmin; dnf install webmin. Somehow people have interpreted me saying to use dnf reinstall as the same as dnf remove/install.)
I’m guessing the module.info files are missing or empty, somehow, and this will maybe fix that.
If Virtualmin modules are among the ones not showing up, you probably need to dnf reinstall those, too.
I should have said I ran only
dnf reinstall webmin
I stopped there as some basic items are still missing from the interface.
I do have other Pro systems which should all be on the same webmin/virtualmin versions. No other Cloudmin installs. I’m saying this in case I need to compare files in some area or copy any over.
version=2.202
longdesc=Setup scripts to be run at boot time from /etc/init.d or /etc/rc.local.
name=Boot Manager
os_support=solaris *-linux hpux freebsd unixware openserver openbsd osf1 irix aix macos netbsd windows
readonly=1
desc=Bootup and Shutdown
syslog=1
category=system
depends=proc inittab 2.202
[root@titan bind8]# cat module.info
version=2.202
longdesc=Create and edit domains, DNS records, BIND options and views.
name=BIND
os_support=*-linux solaris hpux freebsd osf1 irix unixware openserver macos openbsd aix netbsd windows
readonly=1
desc=BIND DNS Server
syslog=1
category=servers
depends=servers 2.202
is from the bind module found in /usr/libexec/webmin/bind8
I don’t have a folder for Bootup and Shutdown in that webmin directory. In the init directory, I only have the one module.info file in the init directory, other than all the ones for other languages with different file extensions.
Looking at another Pro system, this seems to be the case.
I’m asking to see a module.info from a module that isn’t working. If BIND is one of the missing modules, that one provides what I wanted to see…but, if BIND is visible it does not.
So, I believe I know that Bootup and Shutdown is missing, so if I could see the module.info in the init directory, I will be able to rule out missing or corrupted zeroed out module.info files.
I’ve been ignoring issues in /etc/webmin as a source of trouble, since you said you’d compared config files to your backups from before problems? I’m running out of possibilities for why you wouldn’t see all the modules when logged in as root. It pretty much has to be either the package itself is missing stuff (which dnf reinstall would have fixed), or something in the configuration files. It would usually be something missing in webmin.acl and root.acl, but if those match your backups from a working past, we need to dig into the individual module configuration files. So, in addition to showing me /usr/libexec/webmin/init/module.info, also show me /etc/webmin/init/config and while we’re at it, make sureinit appears in the root line for /etc/webmin/webmin.acl and maybe also show me that line and the contents of /etc/webmin/root.acl
That’s pretty much everything that can lead to the behavior you’re seeing, unless there are errors in the miniserv.log that explain why some stuff can’t execute/load and thus some modules can’t load.
Sorry Joe. I should have included the command to see the file in my post above.
[root@titan /]# cat /usr/libexec/webmin/init/module.info
version=2.202
longdesc=Setup scripts to be run at boot time from /etc/init.d or /etc/rc.local.
name=Boot Manager
os_support=solaris *-linux hpux freebsd unixware openserver openbsd osf1 irix aix macos netbsd windows
readonly=1
desc=Bootup and Shutdown
syslog=1
category=system
depends=proc inittab 2.202
Yes, Bind does not show. The above is good information.
I only compared /etc/webmin/virtual-server/config with backups.
In theory the ext3 system doesn’t erase the original until the new is complete. Supposedly, IF I remember correctly, the last thing it does is set the flag on the old to mark the space unused.
:::::::0:0::: is an example from one of these.