Migrating 9 domains from a single box to 3 separate cloud "instances"

Cloudmin
1

We are planning to migrate 9 domains currently administrated under VirtualMin on a dedicated box at GoGrid to three different instances at Digital Ocean

Domain 1 to a separate “droplet”
Domain 2 to a separate “droplet:”
domains 3-9 to a hi-volume “droplet”

After setting up the account(s) we will be sitting there with 3 new droplets running a shiny new Ubuntu OS (we are moving from CentOS to Ubuntu as my developers are not that happy with “down stream Red Hat”)

In the past I’ve done this job myself. VirtualMin helped because you could back up an entire domain, move it to the new box (we used to do this on the LAN inside the GoGrid shed so it was fast) one at a time. But still it was a process fraught with issues. Now we are not moving to a new box inside the same data center but to a new service and separate “servers” (aka “droplets” in Digital Ocean speak)

'm not sure the best way to proceed. I mentioned this here earlier this year and someone said Cloud Min could help us. So what do I do? Does it go like this (please amend, edit as needed, this road map)

After all three servers are up… with shiny new OS but no content. (we will check to include mySql, PHP, PostGre, PostFix for mail, FTP etc… apps that we need) then what?

  1. install CloudMin on one of these?

  2. Give CloudMin access to server (as root?) and then “tell” it to migrate the entire domain (everything /home folder/ mySQL databases etc) to the new “droplet”

what about the one server that will host 7 domains? I have a virtualMin license for 10 domains… can I use this on the three different droplets? If the total domains does not exceed 10?

Do we need CloudMin for physical devices or CloudMin for 10 instances?

Thanks… IMHO the above is not really a road map… and I will appreciate a “real” road map to getting this done.

2

I dont think you can install cloudmin on a VM to then control other VMs.

My host offers cloudmin out of the box to control my VMs. But I still have to manually create domains on a chosen VM. Only cloudmin master admins have the option to assign hosts/domains to the least occupied machine, or the most free memory, etc.

So I think best bet is to have 3 separate virtualmin installs. Backup and import each domain.
Then under webmin server index you’d have 1 click management of each VM.

3

Actually you can use Cloudmin to do what you want but I don’t recommend the Cloudmin for Physical Systems, I tried it and it didn’t do certain things I had to do for some reason, missing features I guess. But Cloudmin 10 Instance Annual License worked great. That’s what I have and I set up a whole bunch of servers on Digital Ocean with it. Works great!

I also have multiple sites hosted per 512MB but I have separated DNS to 2 separate servers of its own and next phase is mail and then MySQL onto separate servers of their own. You will have to tweak and play around with things quite a bit if you go that route too. I also used SaltStack to provision and manage things which has it’s own master server along with some other things on there. I even automated installing Virtualmin and setting everything up like creating the swap drive (make sure you make one of those), the firewall, mail, Apache SSL ciphers, etc. Also may want to look into OSSEC or something for intrusion detection.

Oh and for putting the DNS, MySQL (or Percona), and mail onto their own servers I had to get a thing called Cloudmin Services which I think is some beta software they are working on.

Sorry, another edit, also you can import your backups through Cloudmin onto the system of choice once you get things setup. That is one thing Cloudmin for Physical didn’t do for me due to a missing file, which may have been a bug but there were other things as well I was having issues with. If you want any help feel free to contact me with any questions.

Ok I’m horrible. Final edit, actually answering your road map question, simply make an Ubuntu 14.04 droplet or whichever, make a swap drive, install virtualmin, install cloudmin, thing should be ready to go. You can do any custom tweaks and set things up.

I obviously lied. Note you only have to put Cloudmin on the master system.

4

@rrhode re: multiple edits ha! :slight_smile: I appreciate your spirit/persistence … right… I thought the CloudMin for 10 would be the way to go. I’m actually thinking now of going with Linode and not Digital Ocean because I think Linode has better support. I get nervous with 48-72 hour response times even if it pre-sale queries and on the forums plenty of DO users are not happy with DO support.

But it looks like the scenario/roadmap would be the same.

Our DNS is hosted by a 3rd party. And I not sure we need mail or MySQL on separate servers… our load is pretty low.

how big do you make the swap drive? over here;

http://askubuntu.com/questions/327186/creating-a-dedicated-swap-hard-drive

the recommendation is to make a swap file… not a drive

but then Ubuntu docs on swap all talk about “partition” This is all new to me… I’ve been on CentOS for years now.

https://help.ubuntu.com/community/SwapFaq

On the “master” instance I’m planning to get 16GB RAM (8core) plan, so I guess the recommendation is 32 GB swap drive… sure seems like over kill!

5

I was making 4GB swap files. Make sure whatever you use doesn’t already actually have a swap, not sure about Linode as I’ve never tried them yet but so far DO has no swap by default.

Yeah no need to make a huge swap drive with that much RAM. You probably could put all sites on a server with that much RAM no problem unless they are doing something with all that memory. But good to keep them separated too if you are using one for dev or someone wants their own system for whatever reason or something like that.

I have about 8 to 10 512MB servers, some with probably as many as 8-10 sites on them, and some 1GB servers as well as a 2GB game server. Things seem to be working just fine so far most of the time. I had to tweak Apache and PHP after importing and make sure the php.ini for each site has the new settings since coming from CentOS importing backups will bring their php.ini which will be missing some things like the opcode and some other little things. I just diffed them and copied the changes over

https://www.digitalocean.com/community/tutorials/additional-recommended-steps-for-new-ubuntu-14-04-servers#create-a-swap-file

The 3rd tutorial of that series has a section on creating a swap file that has a few commands to run that makes one.

The full series starts here: https://www.digitalocean.com/community/tutorial_series/new-ubuntu-14-04-server-checklist

I’ve actually also been using CentOS for years but decided to give Ubuntu a shot and don’t regret it once I got used to the commands. There are a lot of newer packages available that seem to get updated more often and faster as well which is usually pretty great. I like how things are so far.

I actually have now begun the process of converting from basic Virtualmin setup with Apache/MySQL to an nginx/Percona/Varnish setup as well and things are somewhat time consuming and tricky at first but I’m starting to figure it out. I can’t imagine doing this as (mostly) smoothly with anything else besides Virtualmin/Cloudmin and the support of those guys.

I considered Linode as well as another one, VULTR or something like that and maybe a few others that were too pricey. But I’m liking the simplicity and ease of using DO so far. Will probably have to try Linode some day too. I liked the DO docs as well.

I normally don’t use support from the hosting company too often but if there’s a hardware issue and a server goes down and they take too long to put it back up I will just move it over to Linode or another one and it should only take a few mins. What else is great is with SaltStack I could just as easily make a new Linode server with a little command and provision everything there instead.

Also had some weird IP issues where sites had their own dedicated IP addresses had to go edit those and set them to the shared system IP. Also had to sometimes patch the postfix config files up which I think was from on CentOS when they had copied the SSL cert to their mail and it messed that up a bit. Also had to set some settings I think in the Apache module config to make new SSL sites use * instead of an IP address in order for SNI to work properly.

That is another thing with DO I believe you can only get 1 IP address per machine and sometimes if you destroy it and launch it again you just may have a different IP. Luckily I figured a way around that when automating provisioning of them for syncing databases from remote services and whatnot.

6

re CentOS / Ubuntu… one of my developers (in Brazil) likes Ubuntu because the upgrade process is more transparent. With CentoOS running the same point OS number and then running “future” patches from later RedHat… etc… you never quite know where you are… You have to run rm on Yum installation logs or something like that… to see if patches are good for PCI compliance audits. But Linode keeps their OS PCI compliant for you (DO does too, I think)

Linode pre-sales responded to my query in less than 1 hour from the time I sent it Hawaii
standard time… I like that… meanwhile 48 hours later I’m still waiting for a response from DO on set up.

Thanks for all the tips. I’ll watch for those issues. On this roadmap:

“simply make an Ubuntu 14.04 droplet or whichever, make a swap drive, install virtualmin, install cloudmin, thing should be ready to go.” Earlier you mentioned that " also you can import your backups through Cloudmin onto the system of choice once you get things setup. "

So CloudMin does not actually import/install the VirtualMin as such? The bigger question I have is: to setup the mail, mySQL, PostGreSQL etc normally you would create the server first VirtualMin and then migrate… is that how it is done with CloudMin?

and, one last question: (was thought VM support would answer this:) I have a VirtualMin 10 license. Can we install that across three cloud instances if we stay <10 domains overall?

7

"So CloudMin does not actually import/install the VirtualMin as such? "

I think maybe Cloudmin could run on it’s own but not really sure, I installed Virtualmin first which installs all the MySQL, Postfix, Dovecot, etc. like normal and then I installed Cloudmin onto the master system. You only have to install it once on the master system and then connect to all the host systems.

Once installed Cloudmin will add an extra menu up at the top where normally you would see Virtualmin | Webmin now it will have Cloudmin | Virtualmin | Webmin. On the Cloudmin menu there is an item near the bottom New System > Add Physical System. You can type in your info here to for the other freshly installed Virtualmin systems that you want to connect up to and then it should be able to connect into those systems from Cloudmin.

Then you can do things on all systems like install/update software, reboot, view the status/system info, transfer files between them, restore backups to them (Cloudmin > Virtualmin Settings > Virtualmin Domains > Restore backup), create new sites, etc. Note that at this time the backup functionality within Cloudmin > Backup and Restore is for VPS systems only and not for Physical systems and you still have to go into Virtualmin to create your scheduled backups like before.

“and, one last question: (was thought VM support would answer this:) I have a VirtualMin 10 license. Can we install that across three cloud instances if we stay <10 domains overall?”

No you cannot, yet. I read they may be adding this in the future. I switched my Virtulamin installs to GPL from a single Pro license and got Cloudmin 10 instead. I’m going to have to program something custom into my billing panel in order for people to be able to manage their script installs and automate installing a new WordPress site for example. But that would likely have had to be done anyway once I convert everything to use nginx. They can still login as their domain user and manage mail and whatnot though. There are a few other features of Virtualmin Pro I miss but at least nowadays the fcgi stuff is setup in it, which works great with nginx BTW.

8

Hmm, so if we cannot install VirtualMin on the two other instances (each of which will host a single site, PHP CMS one is OC Port and the other is XOOPS and both need MySQl)

Can we manage the entire migration, settings, application layer etc from CloudMin for those two instances?

9

You might be fine using Virtualmin GPL on them since you are only having one site one on each one. If you find that doesn’t work for you then you could always upgrade to Pro or wait until/if they setup cross server licenses. They said something about them being slightly more expensive I believe though as well.

In fact I believe Cloudmin only requires Webmin to be installed on the system to be able to communicate with it and Cloudmin should actually offer to install Webmin itself if it doesn’t find it there.

But all you should have to do on them is install Virtualmin GPL. Then connect to it from Cloudmin and then you could try to restore the backups from the old server to those servers under Cloudmin > Virtualmin Settings > Virtualmin Domains > Restore backup.

However make sure you don’t backup the resellers accounts when you initially create the backup because those won’t restore into GPL. In fact you may not wish to restore any of the Virtualmin settings since you would be going from Pro to GPL. Also go through all your configs like Apache, php.ini, mail, etc and make sure to merge/refresh them to the new versions of that software that comes with Ubuntu as CentOS has older versions than Ubuntu 14.04 has which I am guessing will be what you go with.

10

Yes, we would go with the latest Ubuntu settings everything.

For the two domains with only one PHP web site:

  1. spin up the cloud instance, for each one, call them DomainA and DomainB on the dedicated server in SF

  2. install VirtualMin GPL on both those instances…

  3. from the third “Master” site with Cloud min on it… we migrate the old domains DomainA and DomainB from the server in SF to their new instances at Linode West (wherever that is) but if don’t use a backup… a how does CloudMin do the migration?

Sorry to keep plaguing you… but "However make sure you don’t backup the resellers accounts when you initially create the backup because those won’t restore into GPL. " make me wonder how the migration is accomplished if not from a back up/restore process.

I think I just need to go for it and set up the master instance and migrate a few of the small non-sensitive domains that have no big timing issues or content… time to get hands dirty :slight_smile:

11

I just mean in Virtualmin Pro when you do the backup don’t select to backup reseller accounts because those cannot be imported into Virtualmin GPL when you try to restore them there. Virtualmin GPL has no reseller accounts support. Maybe you werent even using that feature anyway.

12

" Virtualmin GPL has no reseller accounts support. Maybe you werent even using that feature anyway."

Right, we were not… only managing our own domains…

Thanks so much for you patience and attention to this. I feel more confident now. We’ll spin up new accounts on the 1st (I like to keep billing/fiscal month on the calendar month) and have some fun!