Help how 9999 owner

SYSTEM INFORMATION
OS type and version debian 12
Virtualmin version 7.30.4

what is it 9999 owner

Is he a hacker?

A successful hack would have root. So, relax? :wink:

I don’t have it but I don’t use Nginix.
Try this from a terminal.
grep 9999 /etc/passwd

Yeah, that looks very questionable. And, it doesn’t look related to Virtualmin. (It appears to be running something out of /var/www, which no Virtualmin domain owner user could write to. So, the user has seemingly escalated in some way, maybe came in through an Apache exploit…have you installed mod_php? That’s one way a user could get access to the server as the Apache user through PHP-related issues.)

s6 seems to be an init system, which most normal users wouldn’t need. But, I guess if you have a user running a complicated web app, it might be OK. It appears to be running some sort of pretty complicated PHP framework/app.

9999 isn’t magically meaningful, but normally that would be a user name if there’s a user in /etc/passwd for that UID. So, do you have a user or UID 9999 in /etc/passwd?

So, short answer: If you don’t know what all this stuff is, then yes, you’ve been hacked.

Ideally, you’d figure out how they got in. Maybe make a snapshot of that system before shutting it down, so you can mount it read-only and poke around in all those various files, and maybe look at the .bash_history for that user, if one exists. And, see what they’re running. There’s a lot of stuff there.

They almost certainly didn’t get in as a Virtualmin domain owner user, though, if they’re working out of /var/www (unless they got in and then escalated due to a kernel bug or some other problem…chroot jails could theoretically be exploited to escalate to root on Debian if you customized the chroot, since the Jailkit package on Debian doesn’t use capabilities and is setuid, but that’s unlikely…I think they probably didn’t start with a Virtualmin domain owner user).

You’ll want to look at a full process list from the command line to see if you can see the full paths of some of the stuff that’s running (ps aux). If they’re really running out of /packages/ they’ve escalated root and you can never trust this system again. Only root could create a directory in / and put files there. So, you can never know what else they’ve done. I’m surprised you see them at all. If someone gets root, they can install a rootkit that hides everything even from root and you’d only ever see them in the form of spam or other attacks originating from the server. The fact that you see them means they’re incompetent, but you still can never trust the server again. You’ll need to start fresh and restore from last known good backups, and make sure you’re up to date.

I found this, which doesn’t really sound like your situation, but maybe you installed some other kind of app/docker orchestrator thing that uses UID/GID 9999 and uses s6 for its init? I would hope you would know you did that, though, this is crazy complicated stuff. This couldn’t have sprung into existence with a tiny install. It would be a bunch of files and since it’s in /var/www and seemingly in /packages it probably had to have been installed by root or a user with sudo privilege.

https://bbs.archlinux.org/viewtopic.php?id=259216

And, still searching, Plex uses s6 and could run as UID/GID 9999 (or others).

But, once again, you should know you did this. You shouldn’t have a random huge pile of software running with elevated privileges that you don’t know about. I remain pretty certain something terrible is happening (maybe the terrible thing is you’re installing a bunch of software you don’t understand, but surely it’s something terrible!).

Hello
Empty, there is nothing

Hello

Empty, there is nothing

What is the solution?

I know nothing about an Nginx install so I thought maybe it could be normal. If you are running Apache then this is definitely not good.
But… :frowning: Take a look at the first appearance more closely.
image
Avoiding Duo with SSH - SCG.

I’d start with shutting that down.

@Joe Yeah. I was thinking a successful root attack would be better covered. Maybe just some downloaded kit by some script kiddie that doesn’t know anything beyond installing something someone else crafted.

Look into /var/www/html and see what is in that. Maybe look up some file names.

I already told you the solution. Whether you believe me or not is not up to me. I’m not going to hold your hand.

Re-read what I have already said. If you have specific questions about what I said, you can ask them, but I’m not going to repeat myself.

I wouldn’t start with telling the attacker they’ve been spotted. OP needs to figure out what actually happened and prevent the user from getting back in, before starting on doing stuff.

Realistically, if it’s a root-level exploit, the system is dead. There is no saving it. Shutting it down and mounting a snapshot of it read-only is the only good way to figure out what actually happened (and is the only somewhat safe way to get data off of it, if OP doesn’t have good backups from before the attack).

A rooted system is not a “let’s have a leisurely look around randomly stopping and deleting things while the attacker continues to have access” kind of situation. You’re just wasting your time and alerting the attacker than you’re aware of them. (They may be unable to act effectively on that information…we already know they suck at doing what they’re doing. Having root would allow a clever attacker to hide from the actual server owner almost perfectly. But, still, you can’t ever be sure.)

Hello

@Joe @ID10T

The problem is with Docker and Coolify,

Coolify has been stopped and 9999 orders have disappeared.

Thank you all

1 Like

We really appreciate your stopping back to let us know what happened instead of leaving us dangling.

Is Coolify a Docker app? Seems strange it would have spawned so many out of container processes.

1 Like

Hello

Don’t worry, I always tell you when I reach a conclusion what happened and what the solution is.

I deleted coolify after a month of doing my job I will reinstall it

Coolify* is an open-source platform for self-hosting applications and databases. It simplifies deployment, supports Docker containers, and offers built-in CI/CD, making it a cost-effective alternative to platforms like Heroku.

This topic was automatically closed 8 days after the last reply. New replies are no longer allowed.