Help designing a 2 server/2 VMs webhosting solution

Cloudmin
1

Hello, I’ve been trying for some time to setup a “simple” solution for a big problem and failed pathetically. I know for years that a big part of the community is asking for some kind of redundancy/fail over/fast spare or anything else helping a fast recovery in case of hardware/software disasters and this is what i need also. So here it is:

  1. I have 2 physical servers and 2 internet providers, and one spare domain for use (let’s call it mumu.ro). In different VLANS, on the switch.
  2. I need one virtual machine on one of them and one provider, and one other vm for the second ones (let’s call them ns1.mumu.ro and ns2.mumu.ro).
  3. In this virtual machines I need to run Virtualmin Pro on one of them (ns1.mumu.ro) and all the required services; and GPL on the other VM (ns2.mumu.ro, this second one being only a second dns and mail server).
  4. In case of something going wrong with one physical server/provider I will be able to start the coresponding VM from a backup on the other physical server, tune it to the new setup (maybe restore some domains from backups to bring them to date) and go shoping for new hardware…
  5. Problem is - now I have one server in production, with a different hostname (say ns1.lulu.ro), and the other for testing purposes, so at the begining I will be running both VM’s on the same test server ns3.mumu.ro, and then migrate ns1.mumu.ro to the other… future ns4.mumu.ro.
  6. Something is wrong here already: I will be having 4 machines: ns1&2.mumu.ro - virtual and ns3&4.mumu.ro - physical. For now there are up only ns1.mumu.ro, a virtual machine on ns3.mumu.ro physical server (the test/spare one). How will I setup DNS in Cloud/Web/Virtualmin, is there a proper, clear procedure to do it? It’s working fine for now only on ns1.mumu.ro except some PTR record.

In the process I need to move all the domains at my registrar from the ns1.lulu.ro (will be called ns3.mumu.ro) to ns1&2.mumu.ro VMs, install CentOS 6 also on ns1.lulu.ro as 5.8 not working see https://www.virtualmin.com/node/22980 and restore the domains on VM ns1.mumu.ro…

Now I have on the test server (called ns3.mumu.ro a brand new CentOS 6.3, Cloudmin and Virtualmin installed, managed to create a the VM ns1.mumu.ro but I still have some DNS I think problems…

Is this feasible, does it make any sense, can you follow me here and advice in doing the job, do I need to buy (hope neot, only 2 VMs… not wanting to sell any for now) the Cloudmin Pro version?

I am at a loss cause nothing is working, no proper documentation for Cloudmin GPL, no how-tos, no answers on the forum…

PS: when I restart/shutdown the physical system does Cloudmin cleanly stop the virtual machines on it, or just “unplugs” them?

2

Howdy,

I don’t want you to think we’re ignoring you, but those are some complex questions, and I haven’t had a chance to answer all of them yet :slight_smile:

I did want to offer a couple of thoughts though –

I’m not sure that Cloudmin Pro or Cloudmin GPL will help you there – unless you happen to have a SAN.

Cloudmin Pro can offers some nice redundancy options – but that relies on you having your server images on a SAN. So if one server dies, Cloudmin can switch everything over to another server, using the image from the SAN.

If you aren’t using a SAN, you’d want to consider a different option. I often suggest copying the Virtual Server backups to a hot spare, where those backups can be imported each night. And then, come up with a process to switch everything over to the spare when something goes awry – perhaps even by changing the IP address of your nameserver at the registrar to point to the new server.

But with your spare also being your backup DNS server, that complicates things a bit… I was hoping to chat with Jamie to see what our recommendation is there.

I am at a loss cause nothing is working, no proper documentation for Cloudmin GPL, no how-tos, no answers on the forum…

I’m sorry that you’re frustrated, but I think one of the big issues is just that Cloudmin GPL doesn’t do what you’re after… Cloudmin GPL is for someone who has one dedicated server, and wants to run a series of VPS instances on it. It’s quite good at simplifying VPS provisioning on a single server, which is a common requirement in the hosting field.

For someone who has multiple servers, such as yourself, and wants to do interesting things across those multiple servers – Cloudmin GPL isn’t going to help you there. Cloudmin GPL is designed to help with just one server.

If you were using a SAN of some sort, and had Cloudmin Pro, the documentation here describes how you could handle failover:

http://www.virtualmin.com/documentation/cloudmin/vm/failover

Essentially – imagine yourself with 3 servers… a Cloudmin Master server, a web server, and another web server… if web server one dies, the Cloudmin Master sees that, contacts the second web server, and launches the VPS instances from the first web server using the second web server. That works since they’re both reading their data from the same SAN.

However, I’ve added it to my todo list to chat with Jamie and Joe to put together some recommendations on handling failover in alternate cases, especially when the backup is also a DNS server.

-Eric

3

I know you have a lot more questions than just this one – but I wanted to mention that I did speak with Jamie, and regarding having a backup/failover server – it’s not going to work if it’s currently acting as a slave DNS server.

If a server is hosting DNS records for a domain, even as a slave server, Virtualmin won’t be able to import the backups for that domain.

So you would need an alternate system to act as the backup server.

-Eric

4

Thank you very much - I know what enormities I’m asking, was a corporate salesman for years but loving more the tech part :slight_smile: ; esentially, I want to do with open source software what others are paying very, very good money for, and it can be done but is kind of a pain, and you have to read a lot (nagging people on the forums is also part of the answer :slight_smile: )

OK - I know how the solution should ideally be, involving a SAN, yes; I can do that very easy but with a custom build SAN you will always end with one machine and a single point of failure - a disaster waiting to happen for the data itself. So a solution would be 2 SANs and some kind of RAID over ethernet/drbd stuff between them… Wich basically is something with more dependencies and posibilities to mess things up, so the direct attached storage should be a better option.

A NETApp dedicated storage&deduplication solution with dual (really, they are dual, totally separated, cool stuff, saw them) controllers would be fine, but none offered to make me it a present at least for now :smiley:

Now what I’m thinking is that I could replicate (not automatically for starters) the way Cloudstack and Openstack work - except not automatic. They don’t need necessarily a SAN. One would ask - why not use them? Because you need a lot of free servers to start things up (3 in a case, 2 in the other), and I don’t have them, those 2 are in production (1 at the moment). Smaller steps is the ideea here, my addiction to your software also.

OK - enough explaining, start asking.

  1. so ns3.mumu.ro and ns4.mumu.ro, the physical servers, shouldn’t be real DNS servers, all that I need is a subdomain for Cloudmin, something like cloudmin.mumu.ro (your default)?
  2. i will use only the 2 VMs (ns1 for pro and all the services, ns2 for secondary DNS and mail) each one on a physical server, with crossed backups so I can start both of them on only one physical server if the other goes down (that’s the simplest thing I can think of).
  3. I still don’t get it how to set the secondary dns server as this issue is still active https://www.virtualmin.com/node/22811 Jamie helped me remotely, but he didn’t said what he did; I ruined the setup in the mean time and I’m ashamed to ask for help again - won’t do that, you have better things to do!
  4. You should consider doing something with drbd in Cloudmin, or add some module in Webmin… or using the new CentOS/RHEL 6.x cluster toys as it seems that somehow we need to use the other underlaying layers, below Cloudmin, sort to speak.
  5. In case that the Cloudmin master goes down - what happens? This is not a good thing that’s the same problem Cloudstack has - the VM’s will probabily be running but there will be no controll over them, at least not from external administration softwares like the billing software, the users will have no control panel no nothing?
  6. Why not investigate also some deduplicating solutions like Opendedup and add some modules in Webmin for easy management. Fr this would be a breakthrough in your product. Imagine someone like me that has at least 6 total machines, all Centos 6.3 exactly the same setup. Just a thought :slight_smile:
  7. Thank you once again - said years ago - I will buy Cloudmin Pro; but for now just need these 2 machines, physical and virtual working and then maybe I’ll have the courage to sell something.
5

Thank you very much I will try setting things up accordingly. Will wait for some other details.

6

Okay, after re-reading all your thoughts/questions above – what I mentioned in my first post above is correct, Cloudmin isn’t going to be able to assist you in the failover setup you mentioned.

That’s not to say it won’t provide a way to handle that at some point in the future, and there’s certainly some ideas we can explore there, but that’s not a problem it solves at the moment.

I’ve added it to my todo list to work on some failover documentation, so that folks have a starting place for setting such a thing up, even if it means needing to set it up manually.

However, let’s dig into your questions –

so ns3.mumu.ro and ns4.mumu.ro, the physical servers, shouldn’t be real DNS servers, all that I need is a subdomain for Cloudmin, something like cloudmin.mumu.ro (your default)?

Hmm… I’m not entirely certain I understand what you’re asking here. If you were to install Cloudmin onto a system, it’s a Webmin module, similar to how Virtualmin is a Webmin module. So when you log in on port 10000, you’d see not just a “Webmin and Virtualmin” link on the top left, but “Webmin Virtualmin and Cloudmin”.

i will use only the 2 VMs (ns1 for pro and all the services, ns2 for secondary DNS and mail) each one on a physical server, with crossed backups so I can start both of them on only one physical server if the other goes down (that’s the simplest thing I can think of).

That unfortunately won’t work… if you have a server setup as a secondary DNS server for say, example.com – Virtualmin wouldn’t allow you to import backups for example.com, as those would end up causing a conflict with the existing DNS records on that server.

You would need a backup server that’s not acting as a slave DNS server.

You should consider doing something with drbd in Cloudmin, or add some module in Webmin… or using the new CentOS/RHEL 6.x cluster toys as it seems that somehow we need to use the other underlaying layers, below Cloudmin, sort to speak.

Sure, I will add this to my notes. That’s not a little feature, that’s a big feature, but we’re always open to input on what features will be useful to folks. Thanks!

In case that the Cloudmin master goes down - what happens? This is not a good thing that’s the same problem Cloudstack has - the VM’s will probabily be running but there will be no controll over them, at least not from external administration softwares like the billing software, the users will have no control panel no nothing?

This depends on your setup. If the Cloudmin master is on a dedicated system (which is assumed for Cloudmin Pro), the VM’s would indeed continue to run, and all the services (Apache, Virtualmin, etc) would be accessed directly on the VM’s until the Cloudmin Master is rebuilt.

Why not investigate also some deduplicating solutions like Opendedup and add some modules in Webmin for easy management. Fr this would be a breakthrough in your product. Imagine someone like me that has at least 6 total machines, all Centos 6.3 exactly the same setup. Just a thought :slight_smile:

I’m not familiar with Opendedup, I’ll check that out. Thanks!

Thank you once again - said years ago - I will buy Cloudmin Pro; but for now just need these 2 machines, physical and virtual working and then maybe I’ll have the courage to sell something.

For just two machines without a SAN, I’m not sure you’d get a lot of bang for the buck from Cloudmin Pro today.

Now, if neither of them was acting as a slave DNS server – Cloudmin would simplify copying domains between them.

But that task can also be done other ways too, Cloudmin just simplifies it.

I hope that helps answer some of your questions!

-Eric

7

Hmm… I’m not entirely certain I understand what you’re asking here. If you were to install Cloudmin onto a system, it’s a Webmin module, similar to how Virtualmin is a Webmin module. So when you log in on port 10000, you’d see not just a “Webmin and Virtualmin” link on the top left, but “Webmin Virtualmin and Cloudmin”.
No :slight_smile: I was asking (sorry for my weird english) what is the deal with the subdomain cloudmin creates - if it can’t be used as a DNS server? Figured it partly myself in the meantime - none as I stopped the bind server.

That unfortunately won’t work… if you have a server setup as a secondary DNS server for say, example.com – Virtualmin wouldn’t allow you to import backups for example.com, as those would end up causing a conflict with the existing DNS records on that server.
Here is also a misunderstanding, I was merely asking (abandoned the ideea of 4 DNS servers) if I can take a VM from one server and started with no problems on the other (some VLAN issues maybe as they are in different VLANs? So maybe I have to change the IP of the VM?) Was asking in another topic about the different backups you softwares does, and maybe here is the problem - I was thinking clones. Just to have a clone of the other VM on the same physical server, not active, but ready to be started up, does it make any sense? Kind of schema here, nevermind the names, not related to DNS questions:

ns3.mumu.ro (physical server) >>>runs>>> VM ns2.mumu.ro and has an inactive clone of ns1.mumu.ro

ns4.mumu.ro (physical server) >>>runs>>> VM ns1.mumu.ro and has an inactive clone of ns2.mumu.ro

In case of something I will just start the inactive clone of the other VM :slight_smile: on the same physical server; from my mobile phone worst case scenario, hoping that I’m not out for fishing :)) abd bo signal. It sure sounds fine to me. And if you will document this properly I think you’ll se a lot of smiley faces.

Well if one can’t move/clone the VM’s than part of all this fuss is gone, you are right :slight_smile:

Thanks for taking into consideration the other blue-skies-beer-mushroom-thoughts.