I know what most people think about rolling releases (not stable, a lot of support and maintain - which might be very well true). However, just sharing my side of the story, in my experience, I have been happily running Fedora Server for the past 5 years and underwent 10 successful, and most of the time, pain-free distro upgrades, which only took 15 minutes to perform. I realize it will not necessarily going to work for all users but certainly could for some.
We might try to officially support CentOS Stream, and Fedora Server in particular, and see how it goes with these Grade B systems.
Also, this thread on WHT from Post 51 on goes into a bit more detail about what they have in mind, and why. Among the reasons are that they also sell KernelCare, which dynamically updates the kernel to avoid the vast majority of reboots. It’s very inexpensive and well worth the cost in my opinion.
CloudLinux may be an option, if I’m reading correctly that they’ll be maintaining an Open Source and free fork of RHEL going forward (historically, they had quirky licensing that I had concerns with). Way back when CentOS took months to release version 6, we added Scientific Linux support (and even ran our own servers on Scientific for several years) to the installer and repos and such…as long as CloudLinux is extremely similar and binary-compatible, we can probably just add OS detection and some minor tweaks and make it go.
Actually, this change also will break our assumptions about RHEL, so we’ll have to deal with that (we don’t have a lot of users on RHEL, but a much higher proportion of our paying customers use it, so we have to support it). I don’t actually develop on RHEL anymore…I used to have RHEL development VMs for all of our supported versions, but CentOS was so close that it turned out to be unnecessary, I just needed to get the repo names right and it always Just Worked. So, it’s definitely piling a lot of new work on us…and we’ll probably have to find another distro that is a straight ahead rebuild of RHEL. Whether that will be CloudLinux or RockyLinux is currently unknown, but I’m glad it’s on the horizon.
As an aside, IBM is really shooting themselves in the foot with this move. I don’t see how it can do anything other than reduce their influence and marketshare on servers. Short term gains in revenue, but long-term loss. I think they’ve built a time-bomb that will effectively destroy Red Hat Enterprise Linux within a few years.
Igor Seletskiy seems open to working with Greg Kurtzer on this. I think that would be about as good as it gets.
As for RHEL, I used it way back in the day and switched to CentOS shortly after it came out (maybe 2004?). I’d never used RedHat’s support anyway, so moving over to CentOS was a no-brainer.
I think what’s happened since then is that the support community that grew up around CentOS got so much better than anything RH ever provided that the license has little or no value anymore. Yeah, they throw in Cockpit and a few other things, but nothing that justifies the license cost, in my opinion.
I’ve also used CloudLinux in the past and actually prefer it to CentOS for shared servers. Last I checked, however, its Apache and PHP implementation weren’t compatible with Virtualmin. I still use KernelCare, which is unsupported in Virtualmin, but still works just fine. I just wait a day or two before doing non-critical kernel updates.
I agree regarding IBM’s foolhardiness. RHEL already has a declining presence in the industry, even in the enterprise space. You’d think they’d be lowering prices at the self-support level and throwing in a few bells and whistles to lure users over from CentOS if that’s what they want to do. Instead, they’ve inspired a new fork. They’re also giving people who’ve been considering Debian and its children a reason to do so sooner rather than later.
There are issues with the restore. I just switched from CentOS to Ubuntu, restored the backups and the fun began: Apache problems, SSL, permissions, etc… and still not done yet solving all the problems.
Yeah, switching operating systems is not expected to be automatic (I hope we haven’t given the impression anywhere that it is). They aren’t the same in a lot of ways; Virtualmin and Webmin paper over a lot of those differences, but there are still some pretty significant things done differently under the hood.
It shouldn’t be a major ordeal to migrate, but, you also should not expect it to be done in an hour or two and without some reading/testing.
Did you restore all features? Its always a mess doing that, for me at least. Even moving from Centos to Centos, every time I restored ‘all features’ and configs - completely messed up new server. What worked was to restore userfiles/data/db etc but leave core configs as is, meaning new server’s apache, php, postfix etc. core configs are left alone. Try to restore with option tick individual features or options. I had similar issues as you (probably) a couple of weeks back and know the pain .
Hi, no, I didn’t. I have tried it first time and it was a mess :)). The second time, I have just restored the files, databases, email addresses, the original IP assignments and that’s it. After that, I could configure it easier than fixing every error that the restore brought. As Joe said, it is normal, because there are different systems and btw, I’ve never said anything about automatic transfer or expect it. I was aware that it will be painful
Glad you resolved it. The amount of restores I did and messed up - well I dont have any hair left. As you I restored parts, tried different options until one was as good as it gets. Lessons learned though, if I ever migrate again that being different OS or not. I know what to expect and can use a check list. I am happy now that is the main thing
I feel you, I was stuck. stuck again and again stuck. I restored more servers because of the switch from CentOS to Ubuntu At one point I was so close to throw everything out of the window. I thought that I’m the only one who has had this kind of pain Now everything is fine.
I think we should learn from this and keep away from ALL the big companies, being Oracle, IBM, CloudLinux or Canonical. They will always sooner or later make a choice in their financial interest and against the ones of the community. All of them already did, like lots, maybe with the exception of RH this being a first for them?
I am very curious, Virtualmin team: do you see something healthy, like Debian, becoming your flagship? Or Rocky Linux if the project is a success? 'Cause I have nightmares with Oracle and the CloudLinux dudes that for years just ignored you and straight answered me that they will not support Virtualmin in any form… Thay are rather in the cPanel team…
not against the rolling distros; in fact you may as well support Arch or Tumbleweed, and I will second this but please make it a good, free and without attached strings distro.
PS: I converted an Epyc Rome host server that I installed CentOS 8 on, to a Monero mining machine This speaks volumes about my level of trust in RH/IBM. Signed the petition, but I will not hold my breath for anything, lesson learned, time to move on.
CloudLinux is not a big company. It was founded a few years ago by one guy, and now it’s still just a few people. Certainly not in the same category as the others. They are a commercial vendor, however, and so market forces still apply.
We are beholden to customers. I have preferences, but I can’t make people use what I like. But, I think inertia is powerful and it’s likely that some other RHEL rebuild will become the de facto for hosting, just as CentOS has been it for so long.
It may also be that the rolling release proves better than we fear. I think it’s worth considering maybe we will all find CentOS Stream to be a good compromise between a rapidly moving OS and one that is stable/reliable.
You mention Debian as an alternative, but it has always been closer to what CentOS Stream is than what CentOS traditionally was. It simply does not have the longterm stability/compatibility guarantee that CentOS historically did. If you want to keep on top of updates, you have to dist-upgrade every couple of years, and their compatibility guarantee even within versions of Debian is much weaker than CentOS/RHEL.
And, Debian does not have the market power to support us even at our current level. It represents maybe 15% of our customers. Ubuntu is already the clear second place distro among Virtualmin users, and I suspect this will push it closer to the lead (and maybe into the lead…it’s been gaining for years).
There isn’t sufficient market to support development for those, currently. We’re a tiny, tiny, team for what we’re trying to do; four people working part-time with effectively zero budget. We already support more distros than we can really handle…it takes longer than we’d like to support distro upgrades when major things change and there’s a lot longer “we’re still finding bugs” period after it rolls out (e.g. when Ubuntu made up a newfangled network configuration system, or the myriad incompatible changes that have happened with MySQL and MariaDB in recent years).
As it stands, RHEL/CentOS represents about half of our paying customers. Wherever those users go is where we’ll go. If that’s RockyLinux or another RHEL rebuild, that’s fine with me (even if it’s Oracle…I read up on it, and their OSS distro is OK, though I still have severe lack of trust toward Oracle, their distro seems to tick all the right boxes).
The positive thing about any of the RHEL rebuilds is that if they are truly compatible, it is very low friction to move between them. As I mentioned, we migrated all of our servers from CentOS 5 to Scientific Linux 6 to CentOS 7. I did those migrations while sitting in front of the physical servers in colo…took a few hours to do all three. Very minor pains (I had a written plan and checklist that was weeks in the making and had pre-backed everything up over the network, of course, when doing those migrations, and was installing new hard disks at the same time).
Anyway, I think we just have to roll with it. I’m in no position to take on more development work (though I am currently furloughed from my other job due to Covid, so I have more Virtualmin time right now, but I’m also dealing with a lot of non-work stuff so still don’t have a bunch of cycles to spare…and will have to get another job early next year if things don’t work out in terms of funding for the company I’m furloughed from).
CentOS Stream will be easy to support, at least initially. It may prove hard to maintain for us. I’ll be running it on some of our less important servers to get a feel for just how painful, so we can make a decision by sometime in 2021 when all of this comes to a head. And, adding RockyLinux or CloudLinux or Oracle Linux or whatever support is relatively easy, as well, if we see a lot of demand for it.
As Joe said, CloudLinux is by no means a big company. I also wouldn’t be at all surprised if they make more money from KernelCare than they do from CloudLinux proper. It’s only a few dollars a month for a KernelCare license, but it’s running on bazillions of servers.
On a broader sense, you also have to think about sustainability, which Igor alluded to in his posts on WHT. Greg most certainly is capable of forking to Rocky Linux. He’s already shown us what he can do. The bigger question is will we support him? Maybe had the industry supported CentOS more generously, they wouldn’t have sold out to RH in the first place, and we wouldn’t be having this discussion.
Also bear in mind that Igor has a vested interest in a FOSS RH fork because it’s the basis for CloudLinux. It would make a lot of sense for him and Greg to work together on this fork. They’re both brilliant, and they both want the same thing. Whether intentionally or incidentally, I believe they’ll wind up either teaming up or cooperating closely.
As for me, I’m going to wait a few months before doing anything except maybe building (or repurposing) a test box to play with. I probably should brush up on Debian just in case. Maybe I’ll drive down to Micro Center and buy a NUC. I’ve been looking for an excuse to go there, anyway.
I also talked to my biggest client about this today, and he basically told me to do whatever I think is best and he’ll okay it. That’s one of the nice things about a 15-year business relationship.
My preference is to wait and see what Greg and Igor come up with because it will make the migration easier than moving to another 'nix; but if the client needs the upgrade before then, I’m leaning toward spinning up a Debian server (with Virtualmin, of course) for the Web, and leaving the mail where it is for now to avoid the inevitable problems with stinky IP’s and blocklists.
It’s all up in the air right now, though. One thing I’m happy about is that I gave up being an early adopter a long time ago. I’m still on 7, so I have some breathing room.
@RJM_Web_Design OK please stop with “the big company” I was referring to the others. Of course Cloudlinux is nowhere near IBM and such.
As you might notice, people are built differently, and I for example have a profound distrust in projects massively driven by companies. I know, let’s not deal in absolutes as most projects have some financial interest behind them, to a certain degree. That being said, it is a matter of degrees; and RHEL just showed us where they stand. It is an abomination what they did - that is not the way to treat your users by forcing them to swallow the rolling pill or buy a license. Sounds like some kind of blackmail to me. Because it is. And Canonical has a pretty ugly record too in regards of erratic decisions (as viewed from the community side).
Why would I choose a company or the other, when I already know that they will pull some sh(…)
sooner or later? Oracle? F(…) them! Cause I surely didn’t forgot all the Sun era, the MySQL issues, the Java lawsuits, OpenOffice slowly dying, the hardships that ZFS endures to this day; to strong-hand people = their business model. This example being the worst I can think of. And the best, as forks flourished.
So better to keep distance, less drama. That’s why I always try to opt for community driven projects or at least with a very healthy direction (as yourselves) and I am interested in your options/opinions. To picture my problem: I just finished setting a whole university on this dangerous CentOS road to nowhere - luckily not my problem anymore, just feeling weirdly responsible. Also I am using CentOS since 2003-04 alongside Webmin at my very small company, loved the OS.
So I guess I will use CentOS 7 on the course of next year and I will test other OSes as a replacement as suggested. But I will kill CentOS 8 just because I can. Like they did. <<< This is a joke. But I will, nevertheless lol
Thank you @Joe for taking the time to discuss this! Keep up the good work!
Just personal opinion but after this move I’ll find it difficult to use anything connected to RedHat simply because doing what they did this time means they won’t hold up next time. What if a lot of people ditch Centos like I suspect they will? That means distros get the people and community development and RedHat doesn’t. They could then decide a year from now that it’s just not worth supporting Fedora, Centos and Red Hat so I suspect one of the two may be eliminated at some point. Maybe RedHat decided to get rid of both and just keep RedHat and go all pay services only. We used to think we knew but this move shows that they could make another erratic move at any time for any reason.
I have not decided what I am switching to but getting away from the RPM and RedHat systems is the goal to avoid any further problems with them. I tested Debian with a Webmin/Virtualmin install but had a few issues. Tested Ubuntu 20 and had no errors at all during either install and already have three in production running however they are only stream servers. I transferred the backup of one of my smaller website and again no issues at all. Eventually I will test one of my larger sites but since all are PHP/MariaDB websites built on wordpress I am not anticipating any large issues. All the same packages install with the LAMP stack either. The bigger worries are when I get to mail and other non website related items but for everything that I see I think I will be going to Ubuntu since for everything I see they have the most modernized OS… CentOS has been stuck on 3.10 kernel for years. Debian is on 4. Ubuntu is on the most modern 5 and while the support cycles are shorter next to CentOS it’s acceptable for now. To be completely honest I would have had at least a few issues with the CentOS install… so far not a single error of any kind and the only real difference was using apt-get instead of yum but how hard is it to switch from yum install php to apt-get install php? It’s not. The deeper items such as mail, virus and spam protection and DNS servers that I worry about. If those work fine upon a restore then they will be the OS to run with. Luckily I designed most of my websites to be extremely portable and universal - same as the control panel - so outside of a little learning curve this seems the best move because I still remember mySQL and Java being destroyed by Oracle and RedHat just pulled an Oracle. If you do it once you will do it again. The only thing that would pull me back to RedHat would be for them to quickly announce this was a huge mistake and they are going to take a step back and rethink things. That’s the ONLY thing and I don’t see it happening.
I keep thinking at this matter, no matter how much I try to put it aside. I agree with @robinstl68 it is the most dangerous precedent, and the community surely will unleash some kind of viral reaction to this. There is no better thing that RHEL could do to earn back the trust of their users but to step back and I am sure they just won’t. Would be childish to think otherwise.
I think I will opt for Debian, Canonical being a problem for me if I go with Ubuntu. I will reinstall my new server with 10.7 but it is very hard to get out a word from the team regarding Cloudmin, not such an active community like for Virtualmin. I am already aware of an older problem with the newer KVM version and Cloudmin; if they would fix it I will throw in it in production with a few personal VMs for starters. And then I will start new VMs with Debian also, port stuff and such.