You need to separate the concept of load balancing and backups in your plans. One is pretty simple, the other is very complex.
rsync is a great solution for static files…it isn’t safe to use on live databases. Hot, and quick, backups of databases is actually a pretty hard problem…and there are always trade-offs in performance, sync times, etc.
If you want load balancing things explode in complexity. Realistically, your application must be built to deal with multiple databases.
The simplest solution for load balancing is to build a highly reliable data server…RAID, redundant disks, etc. And then use that as the backing store for /home and the databases in /var, mounted via NFS. This allows as many frontend servers as needed…but, of course, the bottleneck then (eventually) becomes the file server. And it also becomes a single point of failure, though reliability can be built it pretty well with modern hardware.
What I’m trying to say here is that redundancy is a hard problem, and load balancing is an even harder problem. Not the actual copying or the load balancing itself; those are easy. IP takeover is also dead easy. The hard part is in making those decisions about how long data can be allowed to be out of sync, how to safely and reliably copy things that can’t be copied “live”, like databases (MySQL has some replication capabilities these days, but they’re still actually pretty newfangled and complicated to setup and not standard in the MySQL distribution), and what you’re willing to spend in time and money to make it happen.
If you search for “clustering” here in the forums, you’ll find a lot of discussion on the topic. The thing to note is that everyone has a different idea about what it ought to look like and which problems are important and how they can be solved for a particular deployment (and you’ll also note a distinct lack of good solutions to the really hard problems…since they’re always very specific to your deployment and applications). This is why there isn’t an off-the-shelf “scale your website” product on the market. It just isn’t possible to do it in a generic fashion…everything has to work together, and the most important component (at least in terms of database scaling and load balancing and such) is your applications, and we have no control over those.
Anyway, if a 72 hour window is acceptable to you as your last paragraph seems to indicate, then you can do a MySQL backup before the rsync run, and copy the dump files to your secondary server, and then restore those dumps. That’d actually be pretty easy. But, you can’t possibly expect to load balance between a master and a secondary that are up to 72 hours apart.
We probably will be introducing a "slave server" feature in the not too distant future that performs backups on the master, and then restores automatically on the slave, and also sets up IP takeover. However, this is nowhere near as good a solution as it sounds like. How do you deal with mail from the interim between data syncs? Messages will be lost. How about databases? Your store may have gotten orders that will "disappear" in the change over. This could be a very serious problem or a very small one, depending on your perspective…but, without application support (or accepting the costs and tradeoffs of having a redundant database configuration), there is no solution to that problem.
Then again, it is a step up from simple nightly backups…and then having to take time to manually restore everything to a new server in the event of a system outage. So, that’s the problem we’ll attempt to solve. Beyond that…I dunno that there is anything we can do…we’re still thinking on it, though.