Disaster Recovery - Ransoware attack

SYSTEM INFORMATION
OS type and version Centos7 - Rocky 8
Virtualmin version not sure - latest

Hello,
unfortunately a Ransomware damaged my boot loader. I instaled a fresh RockyLinux 8 with latest script.
I’m able to read old server’s disk contents, but the problem is that I hve not an up-to-date backup and I’m trying to copy & paste to recovery my server.

I found an old 2015 similar post (Emergency Recovery), but things may changed and I’m not sure if I can simply follow that instructions.

Anyway, as my knowledge is really poor, I’m still fighting with permissions etc.

Also, I have not clear knowledge of paths and folders Virtualmin use.

At the moment I created a new installation with a different server hostname with same IP as old one, and for first I’m not sure if could be better to have the same hostname.
Anyway I really appreciate your suggestions.

Best regards

Really sorry to hear that.
I’m sure you will not like me saying this, but if there is one thing to learn from this (and we all learn at some point - eventually) take regular backups. Though I’m not sure that helps unless you know exactly when the infection happened.

Personally I would trust nothing on the infected disk if the “ransomware” virus has had root access the disk is useless. I would not trust the IP either as it will probably have been blacklisted somewhere. So get a new one and hope it is clean.

The only good thing is to have very strict controls over what is allowed to gain access in the first place. Any ideas where the attack cane from or how it got in?

3 Likes

Unfortunately I catch the ARGSransom or NEVADA one, thant the damaged server is an ESXi.

You’re right on every things, but anyway now I have to recovery what I can.

This ransom damage esxi, but it do only few encryption of vmdk-flat file. It means that boot loader is damaged, but data are available using a liveCD at 99%.

The problem now is that doing a copy and past of contents as per the old post create many issue in booting the new server (at the moment I’m on recovery mode)

There are ways to repair the boot loader which might make finding the initial source of the problem and recovery easier. I just had to do this. My ‘ransomware’ was Windows 8.1 on a dual boot machine with Windows 8.1 final support` update. One final FU from them. I only use that side for my tax software.

This is the way.

2 Likes

Well, I’ve not fount a way to recover the boot loader, I’m not enough skilled in linu or simply I’ve not found reliable procedures.
I can add original disk as secondary or I can boot from a live CD, in every case I miss a good solution to rebuild the boot loader

Virus seems not to had access to the system. It atacks vmware esxi. Than, even if I agree with you, at the moment I need to move datas to a new installation

If the ESXi boot loader is damaged and you are able to access the drive with a LiveCD? Why not just copy or download the VM container that you need? Than import it into another ESXi?

But as the others have suggested you should just fry that drive and start over. Next time take snapshots of your VMs off site.

Is this a home lab or a server in a datacenter?

This ransomware corrupt virtualmachine and of course ESXi. As it does not damage entirely virtual disk, is possible to recover them enaugh to access files and folders, but not make the OS to boot again (the bootloader is damaged).

Than of course I’m creating new server, and Of course I’m not interested in use the old instalaltion.
But I need to recover data, emails and configuration fron the old server.

I simply ask your help in how to do.

Look in the /home directories for each user account to collect the websites /public_html and /Maildir.
You would need to look into where the Databases are stored /var/lib/mysql?

I wouldn’t touch anything else.

Note: Be very cautious on collecting the databases as this could have been where the virus originated from by mysql injection.

Thanks @cyberndt
The old post described a similar procedure. It add a full overwrite of /etc folder that probably is what is too much and make the new server unable to boot correctly.
But at the same time I need to recover all virtualmin configuration (domains, mailboxnames, DB connections, etc).

I suppose that the procedure described in that post is interesting but needs some tuning.

I’ll done a better research but I’m afraid there’s no documentation for this kind of recovery

Or in an email one of the users received … or in a direct post on a website like WordPress … or … need we expand?

1 Like

Hey Guys, please don’t OT.
It’s a Ransomware that put down a lot of ESXi servers from 3 Feb all around the world.

My help request is to recovery a fault server with Virtualmin.
Again, I’ve an old backup but it does not contain latest month emails, and for sure not emails received the day of the attack than I’ve only 3 ideas: being able to make the old disk boot again rebuilding grub - use any disaster recovery procedure that let me rebuild my virtualmin from the old disk without using a backup or use the backup to reconfigure a new installation and than find a way to import data from databases or files.

Sorry but I’m fighting from 10 days with this Ransomware and I need help

@xerse,

Not sure I’d be restoring code that was previously part of a “ransom” attack… Unless you are intending to become a victim again.

Before you recover sites and such, you really need to identify how you became a victim the first time, and address the issue otherwise you’re just setting yourself up again.

2 Likes

I’d create a new server with same OS & Virtualmin versions and create the virtual servers again, then I’d copy the content from old server /home to new server to see if that works.
Before that make sure the files from old server are scanned at least with Bitdefender & Sophos.

If you can restore a backup then copying emails is pretty easy just copy the content of the /home/domainname/Maildir
Be sure to check at least with two to three leading antivirus with heuristics or any other method beyond simple virus signature.

Thanks @jorgecardenas1,

I’ve not a good knowledge of Virtualmin than sorry if I’m asking something obvious.

Are all emails stored only into /home/domainname/Maildir ?
I expected that at least indexes, email folders, email status (read, unread,etc) are stored into the MySQL DB if not all emails.

But by your last sentence it seems that mails are stored as single files into the /home/domainname/Maildir is true?

Does MySQL DB may contains records that let emails to being read correctly by a mail client?

If DB have no rules, it could be possible for me to reimport the last backup I have and simply transfer /home/domainname/Maildir content from the old server to the new one.

Anyway is there a file where I can retrieve the Virtualmin version installed?

Thanks

Which means you don’t know. Which means you shouldn’t trust it.

If I tell you the gun “seems” unloaded, are you really going to point it at yourself and pull the trigger without knowing for sure?

I did a quick search. The one attack, at least, was preventable with a patch and the files are well known. Problem is if the OP has been working on this for 10 days not much we can say that he probably hasn’t seen elsewhere.

Hi Gomez,
Starting from 3rd Feb morning, I and many others are working in checking and analyzing the problem. Many security agency around the world publish in the following days what We already analyzed before: Virtual machines vmdk-flat files are not fully encrypted but only for the first 100 Mb (mainly boot loader) as well as the partition table (but for partition table is possible to correct using testdisk utility).

In not a Virus that catch single virtual servers but a rensomware that encrypt most of files that allow ESXi works correctly.

When I say “It seems” I mean that up to now no one, no security agencies aroud the world have found proof that virtual machine’s are involved or datas where stolen.
Of course even if no one has discovered any security issue on VMs, I prefer to analyze VM by VM and reinstall as much as possible.

My goal is to recovery Virtualmin installation to being able to create a full backup and move on a new fresh installed VM.

I hope it’s more clear.