what is the future of ClouMin?

Due to the problems with HyperVM, I was forced to look for a new solution for our VPS clients ASAP, and I don’t have the luxury of trying out every new product on the market, so I purchased some FluidVM licenses - which was a great disappointing.

So, now I’m forced to look for something new. And this time I want to go the Cloud Computing route. There are many other VPS management programs available, and I’m sure I could eventually find something as good as HyperVM - but with active development & support.

So, my question is this: What is CloudMin’s future? I know we can’t foresee the future and know what’s going to happen. But, what active development is being done, and what other features will you introduce to CloudMin?

  1. when can we see live migration for VPS’s between servers?

  2. When can we see total redundancy & high availability, both for the master server & VPS nodes?

  3. When will we be able to dynamically increase / decrease a VPS’s disk space & memory?

  4. When can see an option where VPS’s are automatically migrated to lesser busy servers, automatically?

All in all, when will CloudMin be a true cloud computing platform? Right now it’s a VPS management system with less features than the competition’s VPS management systems?

Cloudmin is still under very active development, and I have a long list of features that I am working on, such as better HVM Xen support, EC2-style block devices and creating new Xen instances from CD images.

To answer your specific questions :

  1. Live migration for Xen instances is already fully support.

  2. Redundancy for the master is being worked on, and should be available in a month or so. Redundancy for host systems I haven’t looked into yet, but would depend on hosts sharing some kind of cluster filesystem to preserve data if one goes down.

  3. This is mostly already possible - you can change the CPU and RAM of Xen and OpenVZ instances while they are running, change OpenVZ disk limits, and add or remove disks to Xen instances. You can’t resize the root disks of a running Xen instance though, as I don’t know of any way to safely resize a filesystem mounted inside a Xen guest.

  4. We haven’t looked into this at all yet, sorry.

Also, I’d be interested to know what additional features you would look for in a “true” cloud computing platform…

Hi Jamie,

Thanx for the updates :slight_smile:

  1. Really? Cause I haven’t been able to get it working at all. The VM is always shutdown before it’s moved to the new host. The kernel also doesn’t get created on the new server, so the VPS doesn’t startup on the new server. - Look here: https://www.virtualmin.com/node/13103

  2. Is it possible to have this up sooner?

  3. The only option I could see, was to add more cores to the XEN VPS, but then I still need to reboot the VPS to make it take affect - which I suppose it how it works with XEN?

Well, the thing is, Cloudcomputing is supposed to offer higher availability than normal VPS management programs, which CloudMin doesn’t do at this stage.

  1. I’m looking into this bug now …

  2. Multiple master support is next on my TODO list of things to implement, after better HVM support.

  3. You can edit CPU and memory limits at Resources -> Resource Limits.

Hi Jamie,

When I edit the CPU or RAM limits, under Resources > Resource Limits, it doesn’t do anything until the VPS is rebooted. IS this also a bug, or a limitation on XEN?

I’m not very fond of OpenVZ due to the fact that one run-away VPS could cause problems for the whole server. XEN is much more isolated in this regards and doesn’t cause this kind of problem.

Are you sure about that? CPU and RAM limits should get applied immediately - Cloudmin uses the xm sched-credit' command to make CPU changes, and xm mem-set` to change RAM.

yup, I’m sure about it:)

Did you increase or decrease the CPU or RAM? I seem to recall that due to some Xen limitation, RAM cannot be increased without a reboot …

increase.

OK … from memory, Xen doesn’t support increasing the RAM for a running instance.

ok, so with XEN we can’t:

  • increase / decrease RAM without rebooting

  • increase / decrease hard drive space without rebooting

  • adding more CPU cores without rebooting

Are these limitations on XEN only, or on the other virtualization technologies as well? I don’t consider OpenVZ a true virtualization technologies, but rather just a CPU emulator.

So, if we want to have a proper cloud computing environment where clients can dynamically upgrade their VPS’ RAM / CPU / Hard Drive, and even move them around on different servers (i.e. to a server with more free resources, or a server in a different data centre) - then what exactly do we need on our end? i.e. what hardware: do we need anything specific like a SAN, or even a NAS? If a SAN can be used, what protocols do we use? NFS? SMB? iSCSI?
What software, and virtualization technologies do we need? Is CentOS fine, or do we rather need Solaris? And is XEN fine? ( I know XEN best, but could learn KVM / Solaris Zones / VMWare?) isf needed)

And how do we setup CloudMin to work like this.

You can reduce RAM without rebooting, but not increase it - this seems to be a limitation of Xen.

The size of the primary disk also cannot be increased or decreased without rebooting - any, this is a limitation of the way Xen works.

CPU core changes also need a reboot, but you can change the CPU percentage without rebooting.

OpenVZ can make all these changes without rebooting, but as you said it isn’t true virtualization. KVM needs a reboot for any resource changes. Not sure about VMware though.

Hi Jamie,

So, is it fair to say that CloudMin can only be true Cloud Coputing environment (if I can call it that) if it uses OpenVZ? The thing I don’t like about OpenVZ is the fact that all the VPS’s share the same RAM, network, IO & CPU - so if one VPS overloads, or if being DDOS’ed, all the VPS’s on that server suffers.

What about OpenSolaris? Can these options be done, dynamically, on Solaris?

I know VMWare can do it though.

Your definition of “true Cloud Computing environment” does not match any existing cloud computing system.

Amazon Web Services is, perhaps, the de facto standard cloud computing environment, and it can’t do any of the things you’re proposing as minimum mandatory features for “true Cloud Computing”. Resizing an EC2 instance in RAM or CPU requires not only a reboot, but a system migration to a whole new instance! Scalability at AWS is achieved via adding new instances and shutting down instances; not through resizing existing ones. We agree that being able to resize a single instance is nice; and we try to make that available whenever the underlying virtualization tech makes it possible. But, we’re all working within the limitations of the underlying virtualization technologies (AWS uses Xen for their virtualization layer, we happen to support several, including Xen).

The fact is that for scalability and redundancy to work, your app needs to scale beyond a single host; and all existing cloud computing solutions require that capability in your app. Scaling up CPU and memory and disk is certainly a nice feature, but there are hard limits on those resources, so apps need to be able to work with multiple hosts in order to scale beyond those limits (and provide redundancy). So, if the underlying technology supports a mechanism for expanding/contracting an available resource (within available resources on the machine) we’ll support it, too. But, it’s not generally considered part of the definition of “Cloud Computing” as the industry seems to have defined it.

So, where I think we’re weaker than we should be, is in enabling users to expand and contract their usage through spinning up and down instances, and not due to some limitations in expansion and contraction of a single instance. Our API needs to become more general and available so that managing instances is easily automatable and controllable by applications. That’s a problem we can address, and will.

But, what you’re proposing is more along the lines of much better VPS-based hosting. Which is also a worthy ambition…just one that we’re limited on, in terms of what we can achieve, being limited by the underlying virtualization technology and system capabilities as we are (and as pretty much everyone else in the cloud computing management space is, including current market leaders like Amazon). A lot of pieces have to fall into place for that kind of flexibility, and most of them aren’t in the part of the stack we control.

Thanx for the reply Joe.

Although Amazon is the (known) pioneers in this field, I don’t see them as a “true cloud” by these definitions. Their instances doesn’t allow for dynamic expansion / shrinking as needed.

VMWare & 3Tera, on the the other hand have all these features, so there’s a level of expectation from the marker / clients that this is what CloudComputing is about - being able to add more RAM, CPU & HDD space when needed, within the limits of the host systems. But, it’s fairly easy these day to get a server with 24/96/256GB RAM, and upto 40HDD’s as well.

Back to reality though, a lot of people can easily obtain a Dual XEON with 12GB RAM & 4x300GB SAS HDD’s, which is a killer machine, and can host a few large VPS’s very well. Add 4 of these, with a decent iSCSI NAS, and you could have an environment where VPS’s could by upgraded on the fly, as needed. Sure, I understand these limits are in XEN (OpenVZ doesn’t really count for this), but what about KVM / VMWare containers?

KVM is even more limited in this are unfortunately - from what I’ve seen, it cannot even change RAM, CPU or disk limits without rebooting. I haven’t looked into VMware though.

ok, so how do we accomplish easy & “unlimited” scalability with CloudMin?