I found out that my certificates were not automatically renewing anymore. I spent many hours researching how Virtualmin was supposed to renew the certificates and I found out the root cause in the updated issuer for Let’s Encrypt certificates. Looks like you now need to check Organization “Let’s Encrypt” instead of only the Issuer (because Issuer could be “R3” or something else).
After finally discovering the issue and fix, I checked the git repo and… this has already been fixed… looks like I spent so much time for nothing …BUT there is no new version available yet which includes this fix.
@Developers, Please release a new Virtualmin version before many others stumble upon Let’s Encrypt SSL certificates not renewing correctly.
@Users, If you run into this problem, it should be fixed after a Virtualmin update… Or if you want to quickly fix it temporarily yourself… here is what I did in the meantime:
feature-ssl.pl next if ($info->{'issuer_cn'} !~ /Let's\s+Encrypt/i) && ($info->{'issuer_o'} !~ /Let's\s+Encrypt/i);
instead of next if ($info->{'issuer_cn'} !~ /Let's\s+Encrypt/i);
in the apply_letsencrypt_cert_renewals sub
Have you installed the “certbot” app? Doing so will likely address your issue. When Virtualmin detects “certbot” it will use this instead of Virtualmin’s Let’s Encrypt client script.
And when researching the issue I actually ran certbot renew which did renew some of the domain names. However, this seemed to update the certificates located at /etc/letsencrypt/live/but the webserver (Apache) config for all the domains (generated by Virtualmin) are using certificates in their home directories.
As a first try, I ended up setting the SSLCertificateFile and SSLCertificateKeyFile in the Apache configs manually to point to the corresponding certificates located at /etc/letsencrypt/live/ but this didn’t really solve the main issue, because Virtualmin would overwrite that config again with certificates in the home directory when manually editing the LetsEncrypt SSL configuration or when renewing the certificates from the Virtualmin control panel.
So the only real fix for me was to change the feature-ssl.pl script… Because certbot alone might update the certificates themselves, but if Apache is not using them, then it’s not useful to me. I’m guessing the feature-ssl.pl script does a copy to the home directory of the domain or something…
For me, it would also be perfectly fine to directly reference the certificate in /etc/letsencrypt/live/ but then how do I prevent Virtualmin from overwriting that to certificates located in the home directory whenever I edit anything SSL related in the control panel?
You wouldn’t generally run certbot directly, but through the Virtualmin UI or API. I haven’t really dug into the code, but it’s quite possible that Virtualmin sets the output directory or copies a version of the generated cert to another location, while still utilizing the “certbot” app. By running it directly, you are using the defaults set by certbot which could potentially cause it to not integrate as intended.
Anyways, if you’d like I’d be happy to setup a screen sharing session to diagnose the issue with you, and get to the bottom of your issue as LE support in Virtualmin should be working fine.
I presently use Virtualmin on a Debian 10, CentOS 7, and Ubuntu 20.04 server.
I did dig into the code quite a bit while researching this. As a matter of fact, certbot is not even mentioned anywhere at all in the Virtualmin codebase. It is mentioned in the Webmin codebase though, but as far as I understand, those are two different implementations?
I couldn’t find any reference to running certbot through Virtualmin either. Could you please tell me how I am supposed to do that, or where I can find information about that? Because I searched for it, but no results popped up regarding that.
Anyway, I think LE support in Virtualmin always has worked fine before LE changed the issuer (I think around December 2020?) and the issue seems to have been fixed in the yet to be released version of Virtualmin which can be found in the code on the Git repository. This is more of a ‘heads up’ to others that the fix will be released soon if anyone else is encountering this issue…
(The issue was present on 2 different servers running Virtualmin GPL by the way)
I do stand slightly corrected, as I just found out by reading the code a bit more that Virtualmin does in fact call webmin::request_letsencrypt_cert which in turn uses the LetsEncrypt code through Webmin itself (which then does use certbot if it is available).
…But in order to get to that point in the code of Virtualmin, the certificate issuer is checked first, which means it will fail for newer certificates… this also means that it will not try to renew the certificate using certbot (through webmin::request_letsencrypt_cert), because the code to do that will be skipped
If I’m mistaken, feel free to tell me whatever I’m not understanding correctly. I am eager to learn more about how Virtualmin works behind the scenes.
@synio what os and version you have this issue on? I run debian 10 which I’ve upgraded from debian 9 and certbot still working, I had no issues at all. (also running gpl)
This is starting to happen for me too on all of my servers. Auto-renew is failing and I discover this, thankfully, by the expiring soon emails from Let’s Encrypt. Manual renew works without problem. Running CentOS Linux 3.10.0-1160.15.2.el7.x86_64 on x86_64; VirtualMin 6.14; WebMin 1.962. Do have certbot installed, version 1.11.0-1.el7.
Hey Joe, thanks for the news that a fix is coming. Do you happen to have any updates here? I’m asking because we have hundreds of sites across hundreds of Vmin-enabled servers and… a very noisy Nagios server warning us about expiring SSL certificates! We’d sure like to know what to expect re: the 6.15 release and whether we’re going to need to mitigate across all of those servers before then. Thanks again!
Goodness, I sure hope the new version is imminent, as we’d certainly prefer that 100x over making that quick and simple change to the code across hundreds of servers.
I dont know if its related, but the CA root is also missing on many of my servers. The browser works OK, but some people when using services that connect to the website are failing saying there is a authority issue with the cert. I have to manually upload the Lets Encrypt root cert for each site. I dont know what sites its failing without doing a SSL check on all sites.
Today was the first time I noticed this. One of my primary virtual servers expired 0 days ago and I just gave a whirl at Virtualmin → Server Configuration → SSL Certificate: Let’s Encrypt [Request Certificate] and all is well for that one untill the update happens.
Good to have an update coming, I also have issues in several domains that doesnt renew , i do manual update with them, but some other autorenew fine… maybe is because of path where I have certs located? Because I know the path is default for some, but nondefault for others…
…BUT there is no new version available yet which includes this fix.