I think we should learn from this and keep away from ALL the big companies, being Oracle, IBM, CloudLinux or Canonical. They will always sooner or later make a choice in their financial interest and against the ones of the community. All of them already did, like lots, maybe with the exception of RH this being a first for them?
I am very curious, Virtualmin team: do you see something healthy, like Debian, becoming your flagship? Or Rocky Linux if the project is a success? 'Cause I have nightmares with Oracle and the CloudLinux dudes that for years just ignored you and straight answered me that they will not support Virtualmin in any form… Thay are rather in the cPanel team…
not against the rolling distros; in fact you may as well support Arch or Tumbleweed, and I will second this but please make it a good, free and without attached strings distro.
PS: I converted an Epyc Rome host server that I installed CentOS 8 on, to a Monero mining machine This speaks volumes about my level of trust in RH/IBM. Signed the petition, but I will not hold my breath for anything, lesson learned, time to move on.
CloudLinux is not a big company. It was founded a few years ago by one guy, and now it’s still just a few people. Certainly not in the same category as the others. They are a commercial vendor, however, and so market forces still apply.
We are beholden to customers. I have preferences, but I can’t make people use what I like. But, I think inertia is powerful and it’s likely that some other RHEL rebuild will become the de facto for hosting, just as CentOS has been it for so long.
It may also be that the rolling release proves better than we fear. I think it’s worth considering maybe we will all find CentOS Stream to be a good compromise between a rapidly moving OS and one that is stable/reliable.
You mention Debian as an alternative, but it has always been closer to what CentOS Stream is than what CentOS traditionally was. It simply does not have the longterm stability/compatibility guarantee that CentOS historically did. If you want to keep on top of updates, you have to dist-upgrade every couple of years, and their compatibility guarantee even within versions of Debian is much weaker than CentOS/RHEL.
And, Debian does not have the market power to support us even at our current level. It represents maybe 15% of our customers. Ubuntu is already the clear second place distro among Virtualmin users, and I suspect this will push it closer to the lead (and maybe into the lead…it’s been gaining for years).
There isn’t sufficient market to support development for those, currently. We’re a tiny, tiny, team for what we’re trying to do; four people working part-time with effectively zero budget. We already support more distros than we can really handle…it takes longer than we’d like to support distro upgrades when major things change and there’s a lot longer “we’re still finding bugs” period after it rolls out (e.g. when Ubuntu made up a newfangled network configuration system, or the myriad incompatible changes that have happened with MySQL and MariaDB in recent years).
As it stands, RHEL/CentOS represents about half of our paying customers. Wherever those users go is where we’ll go. If that’s RockyLinux or another RHEL rebuild, that’s fine with me (even if it’s Oracle…I read up on it, and their OSS distro is OK, though I still have severe lack of trust toward Oracle, their distro seems to tick all the right boxes).
The positive thing about any of the RHEL rebuilds is that if they are truly compatible, it is very low friction to move between them. As I mentioned, we migrated all of our servers from CentOS 5 to Scientific Linux 6 to CentOS 7. I did those migrations while sitting in front of the physical servers in colo…took a few hours to do all three. Very minor pains (I had a written plan and checklist that was weeks in the making and had pre-backed everything up over the network, of course, when doing those migrations, and was installing new hard disks at the same time).
Anyway, I think we just have to roll with it. I’m in no position to take on more development work (though I am currently furloughed from my other job due to Covid, so I have more Virtualmin time right now, but I’m also dealing with a lot of non-work stuff so still don’t have a bunch of cycles to spare…and will have to get another job early next year if things don’t work out in terms of funding for the company I’m furloughed from).
CentOS Stream will be easy to support, at least initially. It may prove hard to maintain for us. I’ll be running it on some of our less important servers to get a feel for just how painful, so we can make a decision by sometime in 2021 when all of this comes to a head. And, adding RockyLinux or CloudLinux or Oracle Linux or whatever support is relatively easy, as well, if we see a lot of demand for it.
As Joe said, CloudLinux is by no means a big company. I also wouldn’t be at all surprised if they make more money from KernelCare than they do from CloudLinux proper. It’s only a few dollars a month for a KernelCare license, but it’s running on bazillions of servers.
On a broader sense, you also have to think about sustainability, which Igor alluded to in his posts on WHT. Greg most certainly is capable of forking to Rocky Linux. He’s already shown us what he can do. The bigger question is will we support him? Maybe had the industry supported CentOS more generously, they wouldn’t have sold out to RH in the first place, and we wouldn’t be having this discussion.
Also bear in mind that Igor has a vested interest in a FOSS RH fork because it’s the basis for CloudLinux. It would make a lot of sense for him and Greg to work together on this fork. They’re both brilliant, and they both want the same thing. Whether intentionally or incidentally, I believe they’ll wind up either teaming up or cooperating closely.
As for me, I’m going to wait a few months before doing anything except maybe building (or repurposing) a test box to play with. I probably should brush up on Debian just in case. Maybe I’ll drive down to Micro Center and buy a NUC. I’ve been looking for an excuse to go there, anyway.
I also talked to my biggest client about this today, and he basically told me to do whatever I think is best and he’ll okay it. That’s one of the nice things about a 15-year business relationship.
My preference is to wait and see what Greg and Igor come up with because it will make the migration easier than moving to another 'nix; but if the client needs the upgrade before then, I’m leaning toward spinning up a Debian server (with Virtualmin, of course) for the Web, and leaving the mail where it is for now to avoid the inevitable problems with stinky IP’s and blocklists.
It’s all up in the air right now, though. One thing I’m happy about is that I gave up being an early adopter a long time ago. I’m still on 7, so I have some breathing room.
@RJM_Web_Design OK please stop with “the big company” I was referring to the others. Of course Cloudlinux is nowhere near IBM and such.
As you might notice, people are built differently, and I for example have a profound distrust in projects massively driven by companies. I know, let’s not deal in absolutes as most projects have some financial interest behind them, to a certain degree. That being said, it is a matter of degrees; and RHEL just showed us where they stand. It is an abomination what they did - that is not the way to treat your users by forcing them to swallow the rolling pill or buy a license. Sounds like some kind of blackmail to me. Because it is. And Canonical has a pretty ugly record too in regards of erratic decisions (as viewed from the community side).
Why would I choose a company or the other, when I already know that they will pull some sh(…)
sooner or later? Oracle? F(…) them! Cause I surely didn’t forgot all the Sun era, the MySQL issues, the Java lawsuits, OpenOffice slowly dying, the hardships that ZFS endures to this day; to strong-hand people = their business model. This example being the worst I can think of. And the best, as forks flourished.
So better to keep distance, less drama. That’s why I always try to opt for community driven projects or at least with a very healthy direction (as yourselves) and I am interested in your options/opinions. To picture my problem: I just finished setting a whole university on this dangerous CentOS road to nowhere - luckily not my problem anymore, just feeling weirdly responsible. Also I am using CentOS since 2003-04 alongside Webmin at my very small company, loved the OS.
So I guess I will use CentOS 7 on the course of next year and I will test other OSes as a replacement as suggested. But I will kill CentOS 8 just because I can. Like they did. <<< This is a joke. But I will, nevertheless lol
Thank you @Joe for taking the time to discuss this! Keep up the good work!
Just personal opinion but after this move I’ll find it difficult to use anything connected to RedHat simply because doing what they did this time means they won’t hold up next time. What if a lot of people ditch Centos like I suspect they will? That means distros get the people and community development and RedHat doesn’t. They could then decide a year from now that it’s just not worth supporting Fedora, Centos and Red Hat so I suspect one of the two may be eliminated at some point. Maybe RedHat decided to get rid of both and just keep RedHat and go all pay services only. We used to think we knew but this move shows that they could make another erratic move at any time for any reason.
I have not decided what I am switching to but getting away from the RPM and RedHat systems is the goal to avoid any further problems with them. I tested Debian with a Webmin/Virtualmin install but had a few issues. Tested Ubuntu 20 and had no errors at all during either install and already have three in production running however they are only stream servers. I transferred the backup of one of my smaller website and again no issues at all. Eventually I will test one of my larger sites but since all are PHP/MariaDB websites built on wordpress I am not anticipating any large issues. All the same packages install with the LAMP stack either. The bigger worries are when I get to mail and other non website related items but for everything that I see I think I will be going to Ubuntu since for everything I see they have the most modernized OS… CentOS has been stuck on 3.10 kernel for years. Debian is on 4. Ubuntu is on the most modern 5 and while the support cycles are shorter next to CentOS it’s acceptable for now. To be completely honest I would have had at least a few issues with the CentOS install… so far not a single error of any kind and the only real difference was using apt-get instead of yum but how hard is it to switch from yum install php to apt-get install php? It’s not. The deeper items such as mail, virus and spam protection and DNS servers that I worry about. If those work fine upon a restore then they will be the OS to run with. Luckily I designed most of my websites to be extremely portable and universal - same as the control panel - so outside of a little learning curve this seems the best move because I still remember mySQL and Java being destroyed by Oracle and RedHat just pulled an Oracle. If you do it once you will do it again. The only thing that would pull me back to RedHat would be for them to quickly announce this was a huge mistake and they are going to take a step back and rethink things. That’s the ONLY thing and I don’t see it happening.
I keep thinking at this matter, no matter how much I try to put it aside. I agree with @robinstl68 it is the most dangerous precedent, and the community surely will unleash some kind of viral reaction to this. There is no better thing that RHEL could do to earn back the trust of their users but to step back and I am sure they just won’t. Would be childish to think otherwise.
I think I will opt for Debian, Canonical being a problem for me if I go with Ubuntu. I will reinstall my new server with 10.7 but it is very hard to get out a word from the team regarding Cloudmin, not such an active community like for Virtualmin. I am already aware of an older problem with the newer KVM version and Cloudmin; if they would fix it I will throw in it in production with a few personal VMs for starters. And then I will start new VMs with Debian also, port stuff and such.
What were those issues? Report it to me please in PM.
Honestly, I would sign a petition to force apt/apt-get developers to switch to table like output for packages listing! It’s unreadable mess right now, compared to the table like output provided by yum/dnf command.
We would be happy to support and will support any 1:1 compatible forks/clones of RHEL! By the way, RHEL 8 support was fixed yesterday and will work just fine from now on.
Sorry about that. It’s just too much to support. We physically don’t have time for it right now. Considering Cloudmin works decently well with CentOS 7 (which will be supported up until 2024, hopefully), we still have some time to bring better support for Cloudmin on latest Debian and Ubuntu servers. It’s not very difficult to implement, rather very time consuming!
Let me check tonight. I don’t recall what the error was but can check on my test server and for what I remember I want to think it was something that didn’t install right or had issues because the farthest I made it was installing webmin/virtualmin, the default LAMP and Icecast2 so it would be limited to that range and the server wasn’t going to have anything but Icecast running on it - LAMP was installed in case needed later.
A Webmin module for KernelCare would be nice, too. All it would minimally need to do would be defer non-critical kernel updates for a few days (which is what I basically do manually). It could also do more; but since the goal is to avoid reboots due to kernel updates, that would minimally be enough.
I think you have an opportunity to influence these decisions I for one will be very interested in which direction you go or show support for. After getting so much help from you and the rest of the team here over the years, I respect and value your opinion very much on these matters.
For me personally I basically want the equivalent of what we always had with CentOS - a reliable, secure, production-ready distro.
Stream does sound a little bit interesting, as the only thing I wished with CentOS in the past is that it wasn’t so behind in terms of packages - so maybe this could be a good thing and I’m certainly keeping my options open - but as soon as I start experiencing issues with it, or begin to hear that lots of others are, I will be looking to move to something else (such as Debian or Rocky Linux).
Hi Team. Just for the record I’ll be looking at Rocky Linux. I’ve already signed their contribution form and will see what I can contribute to the infrastructure side, which is where I excel.
I had a couple of CentOS 8 machines in test, but will not be pursuing them and will likely just shut them down for a rebuild in the CentOS 7 suite.
I’m glad I didn’t move to CentOS 8 with the companies I consult for, Rocky Linux end of Q2 2021 will likely have a release so by then we can re-evaluate.
That would be pretty tough to pull off for CL because it’s pretty highly-customized, as is Virtualmin.
On the other hand, it would be in both companies’ interests; so I can’t imagine Igor turning down a request by Virtualmin for a few free licenses (nor vice-versa) to see if they could make it work. But Igor’s guys are probably working full-time on Lenix, and our guys are overworked as it is.
By the way, KernelCare works just fine in Virt without an official module. I basically just ignore non-critical kernel updates until they get live-patched, which can take a few hours to a few days. By “non-critical,” I mean not security-related, and not correcting some bug that I’ve actually been experiencing.
Occasionally, the KernelCare patches actually happen before the official kernel updates, in which case I don’t even get the update notification. But a module would still be nice to have.
Here are few things people should know about CentOS Linux Stream:
Stream is NOT clone I used, and kernel will never be as stable as RHEL cloned kernel,
EPEL will not be complied against Stream but against RHEL packages
ElRepo 3rd party drivers will NEVER be compiled against Stream kernels but against RHEL kernels. If you have (hardware) system that needs drivers not in a RHEL kernel, you can loose connection or storage link after reboot.
At some points in time new Stream kernels will come at such a fast pace that driver builders will not be able to keep up
With possible frequent changes in kABI curremt kmod/weak update driver/module model will not be applicable to Stream like it is to CentOS Linux.
but please make it a good, free and without attached strings distro.
This speaks volumes about my level of trust in RH/IBM. Signed the petition, but I will not hold my breath for anything, lesson learned, time to move on.
I was referring to the others. Of course Cloudlinux is nowhere near IBM and such.
It’s unreadable mess right now, compared to the table like output provided by 
I for one will be very interested in which direction you go or show support for. After getting so much help from you and the rest of the team here over the years, I respect and value your opinion very much on these matters.