Partition full, so created space then Usermin ok but Webmin - Failed to start LSB

[hisaltesse]

SYSTEM INFORMATION
OS type and version	Debian 10
Virtualmin version	Latest (1.9?)

I am running a virtualmin virtual server on a hardware node. My hardware root partition got full and the virtual server started acting out… after fixing the issue on the hardware node, the virtualmin VPS is still having issues… webmin would not start, yet usermin is running fine and show emails. My application seems to be running as well.

How can I fix this issue? Are there packages that I need to just reload and how? Applications are running on this server and I cannot afford to reconfigure everything…

Here is what I am getting when I run:

systemctl status webmin.service

webmin.service - LSB: web-based administration interface for Unix systems
Loaded: loaded (/etc/init.d/webmin; generated)
Active: failed (Result: exit-code) since Mon 2022-03-21 02:11:40 UTC; 15min ago
Docs: man:systemd-sysv-generator(8)
Process: 1301 ExecStart=/etc/init.d/webmin start (code=exited, status=255/EXCEPTION)

Mar 21 02:11:40 server systemd[1]: Starting LSB: web-based administration interface for Unix systems…
Mar 21 02:11:40 server systemd[1]: webmin.service: Control process exited, code=exited, status=255/EXCEPTION
Mar 21 02:11:40 server systemd[1]: webmin.service: Failed with result ‘exit-code’.
Mar 21 02:11:40 server systemd[1]: Failed to start LSB: web-based administration interface for Unix systems.

and this command journalctl -xe gives me the following output

`journalctl -xe
Mar 21 02:34:43 server postfix/smtpd[1287]: warning: unknown[XXX.XX.XXX.XX]: SASL LOGIN authentication failed: authentication failure
Mar 21 02:34:46 server systemd[1]: Starting LSB: web-based administration interface for Unix systems...
-- Subject: A start job for unit webmin.service has begun execution
-- Defined-By: systemd
-- Support: https://www.debian.org/support
-- 
-- A start job for unit webmin.service has begun execution.
-- 
-- The job identifier is 756.
Mar 21 02:34:47 server systemd[1]: webmin.service: Control process exited, code=exited, status=255/EXCEPTION
-- Subject: Unit process exited
-- Defined-By: systemd
-- Support: https://www.debian.org/support
-- 
-- An ExecStart= process belonging to unit webmin.service has exited.
-- 
-- The process' exit code is 'exited' and its exit status is 255.
Mar 21 02:34:47 server systemd[1]: webmin.service: Failed with result 'exit-code'.
-- Subject: Unit failed
-- Defined-By: systemd
-- Support: https://www.debian.org/support
-- 
-- The unit webmin.service has entered the 'failed' state with result 'exit-code'.
Mar 21 02:34:47 server systemd[1]: **Failed to start LSB: web-based administration interface for Unix systems.**
-- Subject: A start job for unit webmin.service has failed
-- Defined-By: systemd
-- Support: https://www.debian.org/support
-- 
-- A start job for unit webmin.service has finished with a failure.
-- 
-- The job identifier is 756 and the job result is failed.`

Help would be much appreciated.

[Joe]

Look in /var/webmin/miniserv.error for why Webmin is failing to start.

[hisaltesse]

The file starts and ends like this:

[14/Mar/2022:07:45:31 +0000] miniserv.pl started
[14/Mar/2022:07:45:31 +0000] IPv6 support cannot be enabled without the Socket6 perl module
[14/Mar/2022:07:45:31 +0000] Using MD5 module Digest::MD5
[14/Mar/2022:07:45:31 +0000] Using SHA512 module Crypt::SHA
[14/Mar/2022:07:45:31 +0000] PAM authentication enabled
[14/Mar/2022:07:48:13 +0000] Reloading configuration
[14/Mar/2022:07:48:14 +0000] Reloading configuration
[14/Mar/2022:07:48:17 +0000] Reloading configuration
[14/Mar/2022:07:48:18 +0000] Reloading configuration
[14/Mar/2022:07:50:38 +0000] miniserv.pl started
[14/Mar/2022:07:50:38 +0000] IPv6 support enabled
[14/Mar/2022:07:50:38 +0000] Using MD5 module Digest::MD5
[14/Mar/2022:07:50:38 +0000] Using SHA512 module Crypt::SHA
[14/Mar/2022:07:50:38 +0000] PAM authentication enabled
[14/Mar/2022:07:52:56 +0000] Reloading configuration
[14/Mar/2022:07:53:07 +0000] Reloading configuration
Subroutine list_users redefined at /usr/share/webmin/acl/acl-lib.pl line 42.
Subroutine get_user redefined at /usr/share/webmin/acl/acl-lib.pl line 192.
Subroutine list_groups redefined at /usr/share/webmin/acl/acl-lib.pl line 216.
Subroutine get_group redefined at /usr/share/webmin/acl/acl-lib.pl line 324.
Subroutine list_modules redefined at /usr/share/webmin/acl/acl-lib.pl line 337.
Subroutine list_module_infos redefined at /usr/share/webmin/acl/acl-lib.pl line 348.
Subroutine create_user redefined at /usr/share/webmin/acl/acl-lib.pl line 362.
…
…
…
Subroutine delete_from_groups redefined at /usr/share/webmin/acl/acl-lib.pl line 1749.
Subroutine get_users_group redefined at /usr/share/webmin/acl/acl-lib.pl line 1768.
Subroutine check_password_restrictions redefined at /usr/share/webmin/acl/acl-lib.pl line 1785.
Subroutine hash_session_id redefined at /usr/share/webmin/acl/acl-lib.pl line 1829.
Subroutine hash_md5_session redefined at /usr/share/webmin/acl/acl-lib.pl line 1851.
Subroutine md5_perl_module redefined at /usr/share/webmin/acl/acl-lib.pl line 1908.
Subroutine session_db_key redefined at /usr/share/webmin/acl/acl-lib.pl line 1930.
Subroutine setup_anonymous_access redefined at /usr/share/webmin/acl/acl-lib.pl line 1944.
Subroutine remove_anonymous_access redefined at /usr/share/webmin/acl/acl-lib.pl line 1993.
Subroutine get_anonymous_access redefined at /usr/share/webmin/acl/acl-lib.pl line 2029.
Subroutine join_userdb_string redefined at /usr/share/webmin/acl/acl-lib.pl line 2052.
Subroutine validate_userdb redefined at /usr/share/webmin/acl/acl-lib.pl line 2068.
Subroutine userdb_table_sql redefined at /usr/share/webmin/acl/acl-lib.pl line 2166.
Subroutine used_for_anonymous redefined at /usr/share/webmin/acl/acl-lib.pl line 2239.
Subroutine get_safe_acl redefined at /usr/share/webmin/acl/acl-lib.pl line 2257.
Subroutine generate_random_session_id redefined at /usr/share/webmin/acl/acl-lib.pl line 2271.
Subroutine check_md5 redefined at /usr/share/webmin/acl/md5-lib.pl line 11.
Subroutine encrypt_md5 redefined at /usr/share/webmin/acl/md5-lib.pl line 29.
Subroutine unix_crypt_supports_md5 redefined at /usr/share/webmin/acl/md5-lib.pl line 117.
Subroutine to64 redefined at /usr/share/webmin/acl/md5-lib.pl line 126.
Subroutine check_sha1 redefined at /usr/share/webmin/acl/md5-lib.pl line 137.
Subroutine encrypt_sha1 redefined at /usr/share/webmin/acl/md5-lib.pl line 145.
Subroutine encrypt_sha1_hash redefined at /usr/share/webmin/acl/md5-lib.pl line 154.
Subroutine check_blowfish redefined at /usr/share/webmin/acl/md5-lib.pl line 162.
Subroutine encrypt_blowfish redefined at /usr/share/webmin/acl/md5-lib.pl line 170.
Subroutine unix_crypt_supports_sha512 redefined at /usr/share/webmin/acl/md5-lib.pl line 194.
Subroutine check_sha512 redefined at /usr/share/webmin/acl/md5-lib.pl line 203.
Subroutine encrypt_sha512 redefined at /usr/share/webmin/acl/md5-lib.pl line 210.
Subroutine validate_password redefined at /usr/share/webmin/acl/md5-lib.pl line 220.
Subroutine is_dictionary_word redefined at /usr/share/webmin/acl/md5-lib.pl line 261.
[20/Mar/2022:02:15:42 +0000] miniserv.pl started
[20/Mar/2022:02:15:42 +0000] IPv6 support enabled
[20/Mar/2022:02:15:42 +0000] Using MD5 module Digest::MD5
[20/Mar/2022:02:15:42 +0000] Using SHA512 module Crypt::SHA
[20/Mar/2022:02:15:42 +0000] PAM authentication enabled

[Joe]

None of those indicate a problem. That looks like a successful start.

Have you tried just restarting the service? (And, then look for errors in miniserv.error if it fails.)

[hisaltesse]

I just attempted to restart the server with “service webmin restart” … it gives the follow error and nothing gets written in the ministers.error file.

Job for webmin.service failed because the control process exited with error code.
See “systemctl status webmin.service” and “journalctl -xe” for details.

And the output of those two suggested commands is what I posted in my main post above.

Which is why I am wondering if there is a safe way to reload the webmin/virtuamin modules (I did the LEMP fresh install originally) without affecting current configuration ?

[Joe]

Don’t reach for anything dramatic just yet. It’s counter-productive to try to solve a problem you haven’t even identified yet.

Try:

/etc/webmin/restart

Any errors?

[hisaltesse]

@Joe

/etc/webmin/restart

Yes I get this:

Stopping Webmin server in /usr/libexec/webmin
/etc/webmin/stop: 6: kill: Illegal number: eld_

And still nothing gets written in the ministers.error file.

[Joe]

@hisaltesse

Stopping Webmin server in /usr/libexec/webmin
/etc/webmin/stop: 6: kill: Illegal number: eld_

We don’t care about miniserv.errors, now, because we have an explanation already.

miniserv.pid contains something insane/inexplicable. So, make sure Webmin is not running. You can check with:

ps uax|grep webmin

Kill any /usr/bin/perl /usr/libexec/webmin/miniserv.pl /etc/webmin/miniserv.conf processes you see (there may be a few).

Then remove miniserv.pid (maybe look inside it first to figure out wtf is in it, because that sounds like disk corruption of some sort, maybe from a system crash?). Restarting Webmin will probably work fine after that.

[hisaltesse]

@Joe

Kill any /usr/bin/perl /usr/libexec/webmin/miniserv.p

Yes likely disk corruption… ministers.pid only has the following in it “eld_”

I did it… had a bit of a problem … but I redid it and now webmin started…
Thank you so much for the great support…

Is there any other area of corruption that i need to be concerned about and to address?

[Joe]

@hisaltesse

Is there any other area of corruption that i need to be concerned about and to address?

How would I know? Any file that was open for writing at the time of a crash involving disk corruption can be damaged, and may contain anything or nothing. There are other kinds of failure that wouldn’t even need a file to be open for writing to be corrupt…but, I expect you’d see a lot more problems than Webmin not starting if that were what happened in your case.

[hisaltesse]

I understand… but from a webmin/virtualmin point of view, is there a way then to maybe reload all the modules just in case or would it be better to migrate this webmin to another fresh install of virtualmin in order to be save you think?

Also does the webmin backup also backup the entire server configuration or just the virtual servers?

[Joe]

It’s never a reasonable idea to reinstall a bunch of stuff. You have a package manager, it can tell you what has changed. dpkg --verify (or rpm -Va on RPM-based distros) will give you a list of changed files.

Files in /etc and /var are expected to change (and could not be “fixed” by a reinstall without removing your data and configuration), and maybe kernel modules in /lib if you have some new hardware that needs rebuilt modules. Files outside of those dirs generally should not change from what was in the package at install time.

[Joe]

Also, please start new topics for new questions.

Forum Guidelines: Please read before posting!

In order to maximize our ability to help as many people as possible, and in order to keep things friendly and fun, we have some forum guidelines that we ask you to follow when posting new topics or commenting. I present them below in a delightful series of short “Dos and Don’ts” Do Focus! If you have multiple problems, make multiple posts. Do not combine several problems into one post. Include the following: Distro/version, when you installed, steps to cause the problem, any errors you got, re…