Cloudmin active users survey and discussion

Cloudmin
1

Hello, y’all,

We’re currently reworking and modernizing Cloudmin, and we’d really like to hear from people who are actively using it.

One of the bigger decisions we’re looking at is simplifying Cloudmin around KVM and potentially dropping support for the other virtualization backends, including Xen.

Supporting multiple virtualization technologies adds quite a lot of complexity throughout Cloudmin. Focusing on KVM would let us simplify the code, improve reliability, make installation and management easier, and spend more time improving the features people actually use.

Before we make that decision, though, we’d like to understand what real Cloudmin deployments look like today.

If you’re currently using Cloudmin, we’d appreciate hearing:

  • How are you using Cloudmin? What are the main things you use it for?

  • Which virtualization technology are you using? KVM, Xen, or something else?

  • Roughly how large is your setup? Number of hosts and/or VMs is enough.

  • If you use Xen, are you still creating new Xen systems, or mainly maintaining an existing deployment?

  • If Xen support were dropped, would an assisted Xen → KVM migration tool make moving to KVM practical for you?

  • If you use Xen, do you know whether your virtual systems are HVM, PV, or a mix?

  • Would KVM-only support be a problem for you?

  • How are your VM disks stored? LVM, local image files, NFS, Ceph, SAN, or something else?

  • How is networking set up? Linux bridges, VLANs, routed networking, private networks, multiple bridges/NICs, etc.?

  • Do you use shared storage between Cloudmin hosts? If so, what kind?

  • Do you use Cloudmin’s live migration or failover features?

  • How important are VM templates/images, cloning, snapshots, backups, migration, and automatic provisioningin your setup?

  • What is missing from Cloudmin today? What features, improvements, or fixes would make the biggest difference for you?

  • What parts of Cloudmin do you use the most, and what parts do you rarely or never use?

You don’t need to answer every question. Even a quick description of your setup would help.

We’re also considering whether Cloudmin should provide a supported path for converting existing Xen virtual systems to KVM. If you’re currently running Xen, we’d particularly like to understand what would be needed for you to make that transition.

The goal is to make the next version of Cloudmin smaller, simpler, more reliable, and focused on what active users actually need, rather than continuing to maintain features that may no longer be widely used.

Thanks — we’d really like to hear how you’re using it.

2

I use it daily to manage all of my nodes and VMs

I use solely Xen. I even maintain a repo with the most current builds of it. https://dynavirt.com/

I have 7 nodes with multiple VMs each. All are OL9 hosts. Clients vary from OL, Debian, Ubuntu, Windows

I do as much automated as possible. There are bugs in CM and it’s a bit inflexible with no clear architectural layout of how the guest OSes have to be set up (partitions, drives etc) but I’ve learned that extensively through trial and error. The parts it fails at, I do manually and rely on the rest through CMs automation like backups etc.

I would be absolutely livid and feel betrayed. I’ve used CM with Xen for well over a decade, turned in numerous bugs for fixes and even contributed fixes as well as offered up my template VMs I’ve created to be included in your online image catalog. My licensing while small has been religiously paid without fault all these years. I have FAR too many VMs and too many resources such as my site providing Xen builds to convert to anything else. There’s no reason you can’t leave Xen support in there and just make the bugs a lower priority as you always have.

I use a mix of all of them. I’m a tester for Xen and routinely test different builds of both and find ways to run in either mode.

Absolutely as stated above. PLEASE do not drop support for Xen.

I do all of mine as LVMs rather than files. It just seems easier for me to mount and work with when there’s a problem.

I run two bridges - one for WAN and one for LAN. I assign either one or both to each VM as needed.

Currently I’m using just VMs on each node with RAID 6. I want to switch to shared storage so I can use CM’s clustering option to move VMs from one machine to the other without having to move the LVM/disk from one node to another. Since the LVM/disk would be static, the only thing that would change is the config file being turned up on another server. For this I’m still working to find a networking solution that would be fast enough compared to having the LVM/disk on the node itself. Currently my 8gbps FC is not fast enough. I’m experimenting with 40GB Mellanox cables directly connected from each node to a shared storage server to see if I can get the throughput fast enough to support this.

I can’t use either of these at the moment because I have the LVM/disk on the same node. I’d like to which I just mentioned on the last question once I fix the shared storage issue.

I’m using as mentioned, all custom images I’ve made for CM since the official images are all far too old. I have some for Deb, OL and Windows.

More flexibility in creating images. I’d like to see something where CM can RELIABLY look at a running image and make an image from it that can be reused. I’ve never had this work right so I end up making my own. Again, docs would be great here to explain exactly how they need to be builts so it’s not all trial and error or picking apart the code to see what’s expected as I’ve done over the years.

Create, move, backup

Best of all, I use this with Cloudmin services. All of my DNS servers are CM managed VMs and are connected though CM services to all of my VMs so I don’t have DNS running on each VM for VPSes.

3

Long list of questions: (PS: also some talk of virtualmin as they kinda work together with each other obviously.)

KVM is practical sure.
Xen obviously has some interesting philosophies some argue a better overall security footprint.
XCPng seems pretty cool as well.
Hrmmm don’t store in that manner I essentially just take backups using the software and download them for offline storage.
Yes use networking features.
Shared storage you bet.
Migration and Failover yeah they make sense to have as well as templates/images, cloning, snapshots where I want to get next auto provisioning ehhhh IDK yes and no… Yes to the fact of automation. No to the fact of having subdomains to do it.
Missing from cloudmin ehhh a lot the project has so much potential!

  • I’d like to see an OS image developed on with the security of qubes, and xcpng perhaps and a management console similar to proxmox but have it be a rolling upgrades os like void linux or perhaps suse I think was the other lol yeah I know a bit much don’t take it to heart but we all want this I think at the end of the day.
  • I can’t stress security focus no matter what you do if that means trimming the fat to make it manageable so be it. AI = CVE’s new hehe we went from zerodays to negative days I think lol…
  • Cloudmin is essentially virtualization I keep it on bare metal essentially offline while allowing it to spin up VM’s frankly in a pretty easy manner once your traverse through all the millions of menus and resurect your memory of where stuff is. I like the fact that autoupgrades are relatively easy. Until about the 2 or 3 year mark where you have to do all the OS upgrades would be excellent if this could be done easily on debian with the press of a button really to execute a script to upgrade say from bullseye to bookworm for instance.
  • My only interest in Xen is how it is used for many other hypervisors conversions would likely be equitable from a sales standpoint as something that must be done if you wish to take on real marketshare. That and the security postures.

I’d like to see a security dashboard built into both cloundmin and virtualmin with appropriate warnings you know something smart for folks who may not know where to look a consolidation of features.

Oh and mail server stuff making it possible to have mail if the ISP disables port 25 if possible by partnering with a smarthost or perhaps dynamicdns forwarder to make use of a different incomming port IDK to streamline the mail thing for baremetal.
On one hand it is ballsy to run old school bare metal with open port on the other hand it is also the only way to lock stuff down pretty hard and see traffic without a tunnel and constantly make the hole smaller so to speak.

You can be very specific only in bare metal with a non shared ip dedicated to you but when you buy this they often block the mail port and it would be nice if somehow cloudmin/virtual just made mail possible no matter what. Via the methods discussed. In order to do things like shopping carts for someone I need to buy and lease shared webspace kinda lame I’d rather put that money towards hardware. I guess I could manually configure it perhaps but it would be cool if it just worked so there was no split say for like a contact form on a website vs email difficult problem to overcome and still maintain a proper safe configuration to varying degrees. IDK but it is doable I’m sure.

Yeah IDK otherwise I think the software is worth supporting and I put the money where the mouth is and pay the yearly dues. I definately won’t abandon the project if it went full KVM either.

I guess another way to structure things would be to give cloudmin an extra virtualmin liscense as well so that by default you can make use of it by having access or have single instance virtualmin separated servers that would likely provide better security maybe ish IDK… So 10 separate single servers but ehhh then you may just be wasting resources I can say with certainty but for sure there should be at least 2 VMin instances allowed to be ran with a cloudmin subscription its a little silly that you buy cloudmin I thought at first I was gooing to be able to use cloudmin to spin up virtualmin instances without needed a liscense for each one. But I like having the option to do so even if I can’t or don’t allows for growth.

As mentioned the image repo lags very badly with cleaning old unsafe and adding new images.

4

Hi Ilia

How are you using Cloudmin

Currently not (using Xcp-ng and Proxmox) as cloudmin was not production ready too many tweaks to get it to run on current Debian.

Which virtualization technology are you using?

Both KVM and Xen,

However I do not care which type is settled on, but I am strongly advising pick one only as supporting 2 is to much of a waste of time and resources. KVM is standard within the kernel and would be the one I would run with.

Roughly how large is your setup?

We resell VM’s so lots.

If Xen support were dropped,

A migration tool is good enough. If people have an issue then they can move to something else as we have done in the past because cloudmin wasn’t satisfactory. Yes I know this sounds blunt but welcome to the real world. As we say in Oz I’m not here to f*ck spiders. You cant not be everything to every one.

Would KVM-only support be a problem for you?

No it would not, nor would a Xen only system.

How are your VM disks stored?

Predominately local, but we do use iSCSI as well.

How is networking set up?

Linux bridges, routed networking, multiple bridges/NICs

How important are VM templates/images, cloning, snapshots, backups, migration, and automatic provisioningin your setup?

Very important. Its the basic feature set in all other products.

Thanks
Michael

5

Hi @Ilia,

This is great and exciting news!!

How are you using Cloudmin?

Managing virtual machines on multiple bare metal servers. VMs could be a load balancer, web server, podman server, database server, storage server, log server.

Which virtualization technology are you using?

KVM

Roughly how large is your setup?

5 bare metal, 100s of VMs

How are your VM disks stored?

LVM. I’d really wanted to use iSCSI but the tooling is currently broken. Moving forward, support for Ceph would be preferred.

How is networking set up?

Firewall based NAT and bridging on internal networks. Multiple tagged networks on a VLAN in the data centre. Moving everything to VLANs is planned. Being able to segment clients based on vLAN tags is the way to go IMO.

Do you use shared storage between Cloudmin hosts?

No but we really need to.

Do you use Cloudmin’s live migration or failover features?

No live migration/failover as I use LVM, but they are very useful features to have

How important are VM templates/images, cloning, snapshots, backups, migration, and automatic provisioningin your setup?

Very important. Being able to automatically provision based on rules is awesome.

What is missing from Cloudmin today?

  • Working installer that supports RHEL > 8
  • Scheduled snapshots
  • Support for latest machine types (especially for RHEL 10)
  • Support for deploy scripts for VM orchestration (or perhaps this should just be done with Ansible?), where a fleet of VMs is setup, such as load balancer, web servers, database servers, caching servers, all orchestrated from a script.
  • Improved monitoring (statistics are buggy, I guess due to the amount of data that is produced). Plugging into Loki/Prometheus/Grafana might make more sense?
  • Improve dashboard to show VMs and resources. Make it more data center orientated for fleets of VMs.
  • Improve memory management. We’ve encountered issues with changes being made in Webmin or large report generation, consuming a lot of memory and causing OOM killer to get involved. We can probably manage this better to be honest and have a dedicated server for Cloudmin administration purposes only.
  • Auto-scaling: using clones from a snapshot instead of powering down the VM and cloning (which takes too long).
  • Auto-scaling: Be able to have pre / post actions so we can run bash scripts on scale up/down.
  • Improve Cloudmin Backup so it is more in line with Virtualmin backup features.

Many thanks,

Paul

6

In our current environment:

  1. VM snapshot, move, and clone tools.
  2. Package management for updating hosts and vms.
  3. Webmin linking to manage individual instances.
7
  • I test it but it looks old (hope that get redesign with ui-lib.pl)
  • 40 - 70 VMs per host mostly Ubuntu but I have Alma and Windows server too.
  • I’m using Proxmox but would like to have another alternative to them
  • it looks like the KVM a way of virtualization on Linux (instead of support everything, support one that industry use)
  • LVM, local image files, SAN
  • Linux bridges, private networks
  • I’m using templates and cloning to create or move images to other locations, snapshots before breaking change updates and PBS to store my backups somewhere else also save space
  • I would like to see Cloudmin as Proxmox alternative.
8
  1. How are you using Cloudmin? What are the main things you use it for?
    As intended: a management central point for physical + external and internal VM, all with Webmin installed. Mainly related to hosting domains with Virtualmin.
  2. Which virtualization technology are you using? KVM, Xen, or something else?
    KVM QEMU via Cloudmin; on other machines also, with plain old Virtual Manager, all KVM, mainly for testing or utilitarian stuff. Podman and Docker too. No Proxmox or VmWare.
  3. Roughly how large is your setup? Number of hosts and/or VMs is enough.
  • 2 big a-- hosts, down from like 5 for simplicity, efficiency, power and so one;
  • 2 spare hosts;
  • 1 Cloudmin controller, always wanted a second, sometimes I have one, sometimes not;
  • 11 VMs right now;
  • 2 centralized storage NAS/SANs with ZFS;
  • 1 cold storage in the process of being rebuilt.
  1. If you use Xen, are you still creating new Xen systems, or mainly maintaining an existing deployment?
    No XEN here, though support for other virtualization tech is important. You have to learn, you have to test, you maybe get clients that insist on specific stuff
  2. If Xen support were dropped, would an assisted Xen → KVM migration tool make moving to KVM practical for you?
    As mentioned, for me personally no, no XEN around.
  3. If you use Xen, do you know whether your virtual systems are HVM, PV, or a mix?
    No XEN used.
  4. Would KVM-only support be a problem for you?
    No - but I would love to have options. Nowadays options are at a premium. And I guess it will be more of a problem for you, the Virtualmin/Cloudmin/Webmin team, as the Linux world has a long memory.
  5. How are your VM disks stored? LVM, local image files, NFS, Ceph, SAN, or something else?
    For Cloudmin strictly centralized, exposed via NFS; I can’t stand iSCSI. The nodes can access the same files.
  6. How is networking set up? Linux bridges, VLANs, routed networking, private networks, multiple bridges/NICs, etc.?
  • internet 1Gb/s to hosts;
  • hosts to first NAS storage 40Gb/s direct links, bridged in the NAS;
  • hosts to second NAS 10Gbs for management, backups and failover;
  • VMs communicate with hosts via bridge, no routing, all VMs have only a real, external IP address, not allowed in any other network.
  1. Do you use shared storage between Cloudmin hosts? If so, what kind?
    Sure:
  • for VMs I use NFS strictly, file management is so much better to the point that iSCSI doesn’t make any sense for me whatsoever;
  • for backups, I have SMB mounted remotely on hosts.
  1. Do you use Cloudmin’s live migration or failover features?
    Never worked for me, always wanted to, seems essential. It should be a must.
  2. How important are VM templates/images, cloning, snapshots, backups, migration, and automatic provisioningin your setup?
    Paramount, as I guess for everyone else. Suggestions here:
  • leave alone the check mark " System has been already moved manually", and default it to yes - seems safer because centralized storage should be a thing in virtualization;
  • Cloudmin, on the hosts, when migrating VMs, never cleans the cloudmin-kvm service file for the source host, so more than once the same VMs started on two different systems. Ever tried that :rofl: ? Using Cloudmin from the beginning and to this day I still have this problem. So let’s not talk about migration, when this is not fixed :grinning_face:
  • snapshots: I tend to rely only on ZFS, but I would love to see a mechanism like pause the VM (even stop) and write memory > snapshot > unpause/start > copy the snapshot for a few hours or as long as it takes, as a proper backup solution. Because we are talking files, right? Though doable with LUNs too.
  1. What is missing from Cloudmin today? What features, improvements, or fixes would make the biggest difference for you?
  • good proper graphical console for the VMs, always was a gimmick of some sort, posted about it for years, until I gave up;
  • feedback when creating/resizing as it takes forever, but I understand the need for consistency;
  • getting rid of the way-too-common errors when 2 VMs try to use the same ports;
  • migration I can trust;
  • backup that should be feasible and not take the VM offline forever, see above;
  • we should be able to choose if we want to make a clone/image of a VM as is even if it has Virtualmin, or a clone as an image for future Virtualmin use (this one cleans stuff in the VM);
  • warn me when I will have problems with fstab, fscheck, partitions and so on, that will impede future resizing, cloning, migration etc;
  • resources at a glance maybe, kind of like Webmin has on the dashboard? Another design for VMs, much more lightweight?
  • the creation of a new, image/host/KVM system is a bit confusing for me, always have to poke around;
  • and for heavens sake: where is the documentation, please just take it from archive.org or something - I am saying this with all the love and respect for Cloudmin: please just dump it somewhere on this website.
  1. What parts of Cloudmin do you use the most, and what parts do you rarely or never use?
    Most used:
  • great for seeing if a Virtualmin service is down, the number of domains and such; the column features are super;
  • mass commands and updates are essential;
  • upload a file to a bunch of different systems;
  • obviously the action buttons, mass selects etc;
  • I like the integration with Webmin and Virtualmin; otherwise not much point in using Cloudmin for some people I guess;
  • mind that there are quite a few management solutions out there, so don’t paywall everything or people will resort to Cockpit or Virtual Machine Manager and be done with it; and you will get the Elasticsearch/CentOS/Netdata/Crowdsec/Zimbra/Docker/Proxmox, down-spiral. There is, as you know very well I am sure, a sweet spot for FOSS hardcore fans that you have to hit.
    Never used:
  • for some reason Cloudmin nags me about DNS, domains and machine names > that shouldn’t be a thing. DNS is separate, let’s not mix those (always) together. Or offer an option, to let my machine names alone. Why would Cloudmin care if the name of the VM is a FQDN, mind your own IP I say?
  • I don’t need Amazon, Google and the like - in fact those names annoy the hell out of me, always there in the interface. You do know I can’t stand those companies, right :sweat_smile:?
9

  • How are you using Cloudmin? What are the main things you use it for?
    Webmin links, backups, DNS services

  • Which virtualization technology are you using? KVM, Xen, or something else?
    KVM

  • Roughly how large is your setup? Number of hosts and/or VMs is enough.
    4 hosts, 20 VM, and also a dozen physical systems

  • If you use Xen, are you still creating new Xen systems, or mainly maintaining an existing deployment?
    N/A

  • If Xen support were dropped, would an assisted Xen → KVM migration tool make moving to KVM practical for you?
    N/A

  • If you use Xen, do you know whether your virtual systems are HVM, PV, or a mix?
    N/A

  • Would KVM-only support be a problem for you?
    No :slight_smile:

  • How are your VM disks stored? LVM, local image files, NFS, Ceph, SAN, or something else?
    Local images

  • How is networking set up? Linux bridges, VLANs, routed networking, private networks, multiple bridges/NICs, etc.?
    After creating a VM from a Ubuntu image I always need to manually change netplan config to something like this:

    network:
  version: 2
  ethernets:
      eth0:
          addresses: ['MY_IP/32']
          nameservers:
              addresses: [127.0.0.1, 1.1.1.1, 8.8.8.8, PROVIDER_NS]
          routes:
            - on-link: true
              to: 0.0.0.0/0
              via: PROVIDER_GATEWAY

  • Do you use shared storage between Cloudmin hosts? If so, what kind?
    NFS

  • Do you use Cloudmin’s live migration or failover features?
    No

  • How important are VM templates/images, cloning, snapshots, backups, migration, and automatic provisioningin your setup?
    Important

  • What is missing from Cloudmin today? What features, improvements, or fixes would make the biggest difference for you?
    Integrating with Docker Swarm!

10

Thank you, everyone, for the details provided!

I will update this ticket with progress once a fully reworked and functional Cloudmin 10 is available for development testing.