Hello all,
I’m moving to a new server, better hardware and software, from Centos 5.8, php 5.2.17, MySQL 5.0 to Redhat 6.2, PHP 5.3.10 and MySQL 5.5.22, with double RAM. I have about 80 virtual servers.
How do I backup and restore all virtual servers at once?
I was planning to setup a rsync to sync the home folder prior to migration, so when migrating I would backup everything except home dir. The backup restore will be much faster this way. Is this possible?
Thank you
Howdy,
Although this doesn’t cover migrating home directories the way you mentioned, I thought I’d link to the migration documentation in case you find it helpful:
http://www.virtualmin.com/documentation/system/migrate
You can backup all domains at once, or restore all domains at once – from within Virtualmin -> Backup and Restore. You could also use the command line tools, using “virtualmin backup-domain” and “virtualmin restore-domain”.
You can generate a backup that does not contain home directory data by unselecting the "Server’s home directory and web pages " feature on the Features and Settings page in the Virtualmin Backups screen.
-Eric
Eric explained it perfectly. 
A little addition from my end: It might be feasible to do the Virtualmin backup+restore before you rsync the home directory contents.
I’m not 100% certain, but that might prevent problems, since Virtualmin will try to create the home directory in any case, even if its contents are not part of the backup, when it re-creates domains as part of the restore process. And if the directory already exists from your rsync run, VM might go bonkers.
Since migration is non-destructive on your existing server, you might just give it a try.
Thanks for your comments guys,
My full backup takes about 11 hours to complete (to an external HD), and I want to have as little downtime as possible, so I’m planning to do the following:
1 - Install new server and take it to the datacenter
2 - rsync home dir from the old server with the new server (it will take several hours to complete)
3 - when first rsync is done, stop all services on the old server (downtime starts here)
4 - incremental rsync/home again (this will be several minutes)
5 - meanwhile do a full backup of all virtual servers to a single file excluding home dir, and copy backup file to new server.
6 - after incremental rsync is done, change old servers IP and shutdown old server
7 - change the new servers IP, updating it with the old servers correct IP.
8 - restore the full backup on the new server (without the home dir will not take long. Downtime ends here)
9 - restart new server
10 - test everything
If Locutus is right, I could add 6a step, to move /home dir on the new server to something like /home2, and 8a step to move it back to /home.
That is what I plan to do. If I’m not wrong, the downtime will be less then an hour.No DNS or datacenter firewall rules changes are needed.
What do you guys think? Please send me your thoughts.
…
I’ve just tried to do a full backup without the home dir to check how long would it take, but it failed with the message “A new format backup can only be done when the home directory is included”. What should I do?
Thank you
Howdy,
“A new format backup can only be done when the home directory is included”
For that, you can just change the backup format type.
On the backup screen, where it says “Backup format”, choose “One file per server (old format)”.
-Eric
Hi Eric,
I was choosing the One File Per Server, I’ve changed to Single file and it worked. Full backup of 131 virtual servers on 11 minutes (without the /home dir).
Do you think my steps above will work?
I have some domains that have dedicated IP addresses. Restoring a full backup, will those IP addresses be configured correctly on the new server? What about greylist settings? SSL certificates? Postfix configuration?
What should I manually set up before restoring the backup?
Thanks a lot
However you will migrate, what I did when i had to move to a new server was:
Install the new server, install virtualmin, compared settings with the old server for the stuff I didn’t know by heart. Install extra software if needed.
Once the new server was okay, i moved 1 domain to the new server directly over SSH from the backup feature on the old server.
Then I checked the features again to see if everything was working properly and adjust settings if needed. I checked if the site was running properly via “Preview Website”.
Once I was convinced, I backed up all domains through the backup feature over SSH directly to the new server and restored them on the new server.
Then I checked all domains and adjust IP addresses if needed, moved over the ssl certificates etc etc. until it was perfect. Only then I switched servers. It takes a while but I figured to have it right the first time and control the process manually.
Ronald is right, you can do this migration with barely any downtime actually.
The only things that should change in active domains on a regular basis is database contents, and maybe home directory contents (mostly emails, possibly web files if people/admins upload stuff).
You can install and prepare the new server with Virtualmin like ronald explained, and ship it to the datacenter without taking the old server offline. Then you make Virtualmin backups of the old server (excluding home directories), again without taking it down. Restore those backups to the new server.
Then you do an rsync of all home directories, still without taking the old server down. Rsync can be done on a live system without problems. If files change during the sync, a subsequent run will pick those up.
Then, and also during previous steps, you check if config and restore on the new server are okay.
Only then you take down the old server, and do another Virtualmin backup all of the databases only, and restore those to the new server. Should not take more than a few minutes, since DB contents are usually small compared to home directory.
Then, with the old server down, you do another home directory rsync to pick up files that were changed during the “big sync”. No need to fiddle with home directory names like “/home2” there to not confuse VM.
You can even “rehearse” those steps that mean actual downtime, by doing them with the old server still online. The process will work regardless, only with the distinct danger that DB contents or files change during the transfer.
Check if everything is okay on the new server, then redirect nameserver entries to point to the new server. Oh by the way, one day before the migration you should set the TTL of your domains to like a minute, so that DNS propagation delay won’t spoil your nice almost-no-downtime scheme.
Yes, I’ll do all the steps using a new IP to check everything worked good before the actual migration.
But at the time of the migration I think it is best to use the old server’s ip on the new server, without changing the DNS, because if I change the DNS, while the information is propagating I’ll have both servers online, and I’ll have users on both servers. That can be a problem with ecommerce websites, where users are posting orders on both servers. Also email messages being delivered on both servers, somethings can get lost. Or is there a way to overcome this?
Thanks a lot for your comments guys.
I guess you have a point there. However you may want to close shop for a few hours and migrate at a point with the least customers
Yes, of course. I’ll do it on late hours, after midnight.
I’ll update this post with the results of my tests and migration.
Thanks for all the help.
I’m preparing to move this weekend and from the tests I’ve done some things will have to be moved manually, like postfix configuration.
Where is the Greylisting configuration file so I can copy it?
Thanks
Howdy,
The greylisting config is in /etc/postgrey/.
You would first need to enable greylisting though, that can be done in Email Messages -> Email Greylisting.
-Eric
Hello guys,
New problem on my tests. I have my new server ready with restored full backup (excluding /home dir), and /home dir copied with rsync. All websites are working good, but the problem is all the email folders on IMAP clients have become unsubscribed!
How can I fix this?
Thank you
Where are you Dovecot index files stored?
One way to determine that is with this command:
dovecot -n|grep mail_location
Here it is…
[root@linux03 public_html]# dovecot -n|grep mail_location
mail_location = maildir:~/Maildir:INDEX=/var/lib/dovecot-virtualmin/index/%u:CONTROL=/var/lib/dovecot-virtualmin/control/%u
[root@linux03 public_html]#Thanks
Previous post is for the new server.
For current server it is:
[root@linux03 Maildir]# dovecot -n|grep mail_location
mail_location: maildir:~/Maildir
[root@linux03 Maildir]#To resolve the issue you’re seeing, you could set the mail_location on the new server to use the value from your old server.
Then, copy over the /var/lib/dovecot-virtualmin/ directory (and all the sub-directories), which contains the dovecot index and control files.
-Eric
The current server has 4Gb RAM.
The new server has 16Gb RAM. I’ve used the Percona Online tools to create the MySQL configuration file.
Is there any other configuration I should update to make better use of the extra memory?
Thanks
Hi Eric,
Changing the mail_location did it. All folders are ok now.
Do I really have to copy over the /var/lib/dovecot-virtualmin/ directory ? The /home directory will be rsynced again before the migration.
Thank you!