Load Balancing / Backup Servers

Hello everyone, this is my first post on the forum. I’ve been using Virtualmin for a while now, and have got to grips with most of its functionality etc. However there is one thing I would like to do and wondered if there was a built in function.

I have a main web server which i host all of my websites on running Virtualmin, I have recently purchased a second server which I want to use as a "live backup" of the main server. So in the event of the main server going offline i can simply reroute my IP to the other server, all of the servers/databases etc will be already there without having to restore from a backup.

What is the best way to do this? atm im thinking of using rsync or something similar to basically mirror the main server every 72 hours or so. Any better solutions?


you should be able to do this using rsync with heartbeat.
Some of the webmin’s cluster modules may be of use as well to assist you in this task.
There is a third party module available to handle rsync.
I’m not sure if you’d need the Virtualmin Pro version or that GPL can do this just as easily…

You need to separate the concept of load balancing and backups in your plans. One is pretty simple, the other is very complex.

rsync is a great solution for static files…it isn’t safe to use on live databases. Hot, and quick, backups of databases is actually a pretty hard problem…and there are always trade-offs in performance, sync times, etc.

If you want load balancing things explode in complexity. Realistically, your application must be built to deal with multiple databases.

The simplest solution for load balancing is to build a highly reliable data server…RAID, redundant disks, etc. And then use that as the backing store for /home and the databases in /var, mounted via NFS. This allows as many frontend servers as needed…but, of course, the bottleneck then (eventually) becomes the file server. And it also becomes a single point of failure, though reliability can be built it pretty well with modern hardware.

What I’m trying to say here is that redundancy is a hard problem, and load balancing is an even harder problem. Not the actual copying or the load balancing itself; those are easy. IP takeover is also dead easy. The hard part is in making those decisions about how long data can be allowed to be out of sync, how to safely and reliably copy things that can’t be copied “live”, like databases (MySQL has some replication capabilities these days, but they’re still actually pretty newfangled and complicated to setup and not standard in the MySQL distribution), and what you’re willing to spend in time and money to make it happen.

If you search for “clustering” here in the forums, you’ll find a lot of discussion on the topic. The thing to note is that everyone has a different idea about what it ought to look like and which problems are important and how they can be solved for a particular deployment (and you’ll also note a distinct lack of good solutions to the really hard problems…since they’re always very specific to your deployment and applications). This is why there isn’t an off-the-shelf “scale your website” product on the market. It just isn’t possible to do it in a generic fashion…everything has to work together, and the most important component (at least in terms of database scaling and load balancing and such) is your applications, and we have no control over those.

Anyway, if a 72 hour window is acceptable to you as your last paragraph seems to indicate, then you can do a MySQL backup before the rsync run, and copy the dump files to your secondary server, and then restore those dumps. That’d actually be pretty easy. But, you can’t possibly expect to load balance between a master and a secondary that are up to 72 hours apart.

We probably will be introducing a "slave server" feature in the not too distant future that performs backups on the master, and then restores automatically on the slave, and also sets up IP takeover. However, this is nowhere near as good a solution as it sounds like. How do you deal with mail from the interim between data syncs? Messages will be lost. How about databases? Your store may have gotten orders that will "disappear" in the change over. This could be a very serious problem or a very small one, depending on your perspective…but, without application support (or accepting the costs and tradeoffs of having a redundant database configuration), there is no solution to that problem.

Then again, it is a step up from simple nightly backups…and then having to take time to manually restore everything to a new server in the event of a system outage. So, that’s the problem we’ll attempt to solve. Beyond that…I dunno that there is anything we can do…we’re still thinking on it, though.

thanks for your comprehensive reply, i think the slave server feature is kinda what im looking for. Can’t wait to see it in the future. Until then i think im gonna make a simple perl script to backup the servers using virtualmin api’s. Sync the mySQL with the remote server using the mySQL built-in features and then restore the backups on the other box.

Another option i thought about was making ec2 ami’s every 72 hours from my physical server, which i can then launch on ec2 with all servers intact in the event of failure.

Load balancing isn’t as important for me it seems, rather a fallback solution in the event of failure at a noc.

Thanks again,

this is nowhere near as good a solution as it sounds like.
How do you deal with mail from the interim between data syncs? Messages will be lost. How about databases?

to help reduce downtime while migrating, mySQL databases are normally sync’ed via mirroring/replication - and at least dovecot can be told to use mySQL as storage backend too - so DRBD/rsync isn’t necessarily required.