Multiple Sites Hacked

Operating system Ubuntu Linux 20.04.5
Webmin version 2.001 Usermin version 1.860
Virtualmin version 7.3-1

and

Operating system Ubuntu Linux 16.04.7
Webmin version 2.001 Usermin version 1.860
Virtualmin version 7.3-1
(I’m trying to move all of the sites off of this one and to the above one)

I am using the Webtotem security plugin.

Two sites have been hacked, with insertions of php files, folders, and insertions into wp php files. The files all relate to phishing related to GCash and a Philippine bank. I removed the files yesterday, but they are back on the site I cleaned up, plus a new site on a different server. I am changing all of the passwords, but I have to figure out how they are getting in and inserting this stuff. The two sites are blacklisted by Google. Also one site that has email is sending spam. I have email turned off for now.

This is beyond me - where do I start?

Apache’s logs for the last 24 hours will tell you how the hackers got in the second time.

I use Nginx and not Apache

Well then you should look into your Nginx logs for the last 24 hours…

Here’s a problem. Xenial reached EOL over a year and a half ago. Unless you’re paying for extended support, you have several vulnerabilities in several of the packages you’re running.

That’s probably not how they got in though. It was probably something in WordPress, likely a plugin you’re using. Since that’s where you found the evidence, it seems likely they only have access via PHP as the domain owner user. But, given you’re running a dangerously out of date OS and kernel, you’re likely at risk of a root exploit (though if they had root, you probably wouldn’t see the evidence they left behind).

Anyway, since you’re running old software, I assume you’re running old WordPress and plugins, which is almost certainly how they got in.

That is a good assumption, but I keep all of the WordPress installs, plugins and themes updated as soon as new versions are available. Xenial holes might well explain the one hack, but it doesn’t explain the same hack on the updated server:

Operating system Ubuntu Linux 20.04.5
Webmin version 2.001 Usermin version 1.860
Virtualmin version 7.3-1

Ah, I missed that part. They almost certainly left themselves some additional back doors in your WordPress install, and maybe outside of it as well. Once they’re in, it is very difficult to know with confidence you have really cleaned up.

But, still, they almost certainly got in via an exploit of something in WordPress, probably a plugin. If they didn’t get root, they cannot remove web server logs, which is probably the best way to spot what they did, as calport has suggested. Both access and error logs may contain clues. If you’re using php-fpm, you may have additional useful information in /var/log/php-fpm.