Renewing lets encrypt certificates breaks nginx

BluCobalat · August 26, 2022, 8:54am

SYSTEM INFORMATION
OS type and version	RHEL 8.6
Virtualmin version	7.1 gpl

One of my websites has a lets encrypt certificate that was reaching its expiration but when I renewed it through virtualmin my nginx started crashing with:

nginx: [emerg] SSL_CTX_use_PrivateKey_file("/home/blucobalt/ssl.key") failed (SSL: error:0B080074:x509 certificate routines:X509_check_private_key:key values mismatch)

What I’ve found googling is to check the modulus of the certificate and key with openssl and to see if they match but they do:

[root@web1 blucobalt]# openssl x509 -noout -modulus -in ssl.cert | openssl md5
(stdin)= d258617679803b41352048a6f2800f78
[root@web1 blucobalt]# openssl rsa -noout -modulus -in ssl.key | openssl md5
(stdin)= d258617679803b41352048a6f2800f78

Any help is appreciated. My nginx isn’t starting anymore and the rest of my websites are broken because of it.

Ilia · August 26, 2022, 9:05am

Hello,

How exactly a block of that virtual server (domain) Nginx config that loads certificates look like? Does it look the same compared to other, previously working domains (which weren’t changed)?

BluCobalat · August 26, 2022, 9:28am

from the offending server block in nginx.conf

...
ssl_certificate /home/blucobalt/ssl.cert;
ssl_certificate_key /home/blucobalt/ssl.key;
...

from a server block on another server, with working ssl

...
ssl_certificate /home/cmdbin/ssl.cert;
ssl_certificate_key /home/cmdbin/ssl.key;
...

I tried experimenting with other ssl.* files for ssl_certificate, like ssl.combined, but they all produced the same result.

Ilia · August 26, 2022, 9:44am

This is odd, as it should be ssl.combined in fact. Did you by any change edit ssl.combined manually? Nginx is susceptible to certificates order.

I would suggest the following to fix an issue:

Disable Nginx and Nginx SSL website feature for offending domain (using Edit Virtual Server page)
Restart Nginx so other sites would work
Delete SSL certificates for this domain, using virtual-server.name - Server Configuration ⇾ SSL Certificate page (preferably) or manually but make sure not to restart any other services on the system not reboot!
Next re-enable Nginx and Nginx SSL website features
Unless certificate is automatically requested, request it manually
It should work here

Please do not touch SSL certificates manually.

BluCobalat · August 26, 2022, 9:55am

No, but to make sure, I re-requested the certificate and still received the error. I followed all the steps (including manually deleting ssl.*, not restarting any services or rebooting, and then manually requesting through server config > ssl certificate > lets encrypt) and am still getting the SSL: error:0B080074.

I think what happened was that the server’s lets encrypt configuration got confused with the certificates it had and I manually set the certificate and key in ssl certificate > update certificate and key. I’ve never had an issue with this before until now, as this is what I had to do when subservers occasionally wouldn’t pick up the root server’s certificate. Once thats done though, virtualmin recognizes that the sub server is sharing with the root server.

FWIW, webmin can use the certificate requested just fine

Ilia · August 26, 2022, 10:12am

I have just specifically spun up Rocky 9 with Virtualmin LEMP stack to test it and it worked for me straight forward. LE SSL certificate was successfully requested on the wizard setup stage.

The records set up by Virtualmin were as follows:

ssl_certificate /home/host3/ssl.combined;
ssl_certificate_key /home/host3/ssl.key;

I have hard time guessing at the moment why it’s not working for you.

Do you have all system packages updates installed? What is the output of:

date
dnf clean all
dnf update -y

BluCobalat · August 26, 2022, 10:18am

That’s what it looks like for me after disabling and reenabling nginx {,ssl} website.

When I first encountered the issue, the first thing I tried was making sure everything was up to date and that still seems to be the case:

[root@web1 blucobalt]# date
Fri Aug 26 03:12:44 PDT 2022
[root@web1 blucobalt]# dnf clean all
Updating Subscription Management repositories.
89 files removed
[root@web1 blucobalt]# dnf update -y
Updating Subscription Management repositories.
Sonatype Community                                                                      367 kB/s | 447 kB     00:01
ELRepo.org Community Enterprise Linux Repository - el8                                  114 kB/s | 245 kB     00:02
Extra Packages for Enterprise Linux 8 - x86_64                                          5.5 MB/s |  13 MB     00:02
Extra Packages for Enterprise Linux Modular 8 - x86_64                                  425 kB/s | 857 kB     00:02
Red Hat Enterprise Linux 8 for x86_64 - BaseOS (RPMs)                                    25 MB/s |  51 MB     00:02
Red Hat CodeReady Linux Builder for RHEL 8 x86_64 (RPMs)                                8.6 MB/s | 7.4 MB     00:00
Red Hat Enterprise Linux 8 for x86_64 - AppStream (RPMs)                                 24 MB/s |  46 MB     00:01
Remi's Modular repository for Enterprise Linux 8 - x86_64                               187 kB/s | 1.0 MB     00:05
Safe Remi's RPM repository for Enterprise Linux 8 - x86_64                              348 kB/s | 2.2 MB     00:06
RHEL/CentOS 8 - x86_64 - Virtualmin                                                      25 kB/s |  18 kB     00:00
Virtualmin Distribution Neutral Packages                                                470 kB/s | 690 kB     00:01
ZeroTier, Inc. RPM Release Repository                                                    23 kB/s | 6.4 kB     00:00
Dependencies resolved.
Nothing to do.
Complete!

Ilia · August 26, 2022, 10:35am

Does certificate AN also include bluecobalt.dev or only *.bluecobalt.dev?

Manual explains an error as incorrect chaining of the certificate file.

https://nginx.org/en/docs/http/configuring_https_servers.html#chains