Let's Encrypt SSL fails with 404 error on sub-server website

SYSTEM INFORMATION
OS type and version CentOS 7
Virtualmin version 2.011

My virtual server was created by migrating it from a cPanel account, containing 2 websites. For the “main one”, I’ve already successfully generated the SSL certificate, but for the other (configured as “sub-server”) is failing.

According to the letsencrypt log file, this is the error message I’m getting when running the web-based (http) validation:

Domain: [my_domain]
Type:   unauthorized
Detail: [my_vps_ip]: Invalid response from
http://[my_sub-server_domain]/.well-known/acme-challenge/E0in-9OT2CrNJw6RHpSXEUcTC4OgE-G7CKkxcdXOzxg: 404

As previously stated, since I am migrating the site from cPanel, I simply edited the DNS entries in Cloudflare (which is the DNS provider) to point to the new IP address.

My first attempt to troubleshoot this issue was to check that “.well-known/acme-challenge” was indeed provided in webroot directory, with correct folder and files permissions. Although I can indeed curl text files located in this path, I am not sure whether letsencrypt was using indeed the “correct webroot” path, since under Virtualmin > Server Configuration > SSL Certificate > Let’s Encrypt I can see that the webroot cited there is not the correct one (at least not the one configured in the apache server for that website).

As I didn’t find out from where exactly letsencrypt was sourcing that webroot info, I’ve created the “.well-known/acme-challenge” path in that other folder as well, but this doesn’t seem to be the issue and I still get the same 404 erro on http validation for this website.

Any ideas what could be the issue here? Is there any additional measures to be done for sub-server websites, so that the letsencrypt SSL script doesn’t fail?

Thank you in advance for any support!

You do not need to create that path. It’s done automatically.

But, I’m guessing either the right site is not being served, or you have a redirect or proxy rule that doesn’t allow .well-known path to be served from the filesystem.

You can test this by putting a file in the .well-known dir, just a simple text or HTML file and browsing to it with your browser. Then look in the access_log for that site to see what’s happening. If you see nothing, you’re getting the wrong site (either DNS or Apache VHost misconfiguration). If you see something, maybe there are clues about what’s happening.

@Joe,

You do not need to create that path. It’s done automatically.

Now I see I chose the correct category for my post! :man_facepalming:

You can test this by putting a file in the .well-known dir, just a simple text or HTML file and browsing to it with your browser. Then look in the access_log for that site to see what’s happening.

Yes, I am able to create a file within the path “.well-known” and visit that page through the browser. In the access_log file, I can see the 200 response to that request. Strangely, though, I can see another request to that same URL, but with https (instead of http). And there is also another request to the favicon of the site, which is executed with https.

So it really seems that there is a redirection rule added somewhere. I must also note that I have already deleted everything from my .htaccess file (it only contains the basic/default entries for a Wordpress site), and the Virtualmin is not configured to redirect requests to the SSL site (Virtualmin > Server Configuration > Website Options ---- “Redirect ll requests to SSL site” > NO selected).

Is there any other way that Virtualmin sub-server configuration could be redirecting requests to https? Or waht else am I missing?

Maybe it’s unrelated, but I must also note: after I created the SSL for the “main site” on this same virtual server, I clicked on “copy SSL certificate to Services” (so that I could test the mail server, for example), and it seems that this certificate got copied also to the sub-server website as well (and I don’t know if this could have any impact on this additional SSL generation attempt).

If this was migrated from another system that wasn’t running Virtualmin, it may very well have redirects that Virtualmin doesn’t know about.

Check .htaccess and check the VirtualHost sections for any redirects that don’t exclude the .well-known path.

@Joe

I’ve successfully removed all redirects (by disabling Wordpress plugins), and my SSL certificate generation attempts still result in “404 unauthorized” errors.

I then started to do some testing using the CLI (certbot command), through dry runs, to get some additional insights. And now for me the root cause of the issue is quite clear: the webroot path that Virtualmin Let’s Encrypt uses for trying to generate the SSL certificates is not the correct one. Explained in other words: the result of the dry-run is successful as soon as I change the webroot path to the correct one; when I use a wrong webroot path, the dry-run results in “404 unauthorized” error. So my intuition (explained in my first post of this topic) was indeed right.

So now the issue is: how do I edit this sub-server webroot path so that the Let’s Encrypt script within Virtualmin can run successfully? I thought I could change it through Virtualmin > Server Configuration > Website Options – Website documents sub-directory, but I get the following error:

"Failed to save website options : Website documents sub-directory cannot start with or end with /".

Since I am trying to change that path from “/home/my_user/public_html/sub-server” to “/home/my_user/sub-server”, that error doesn’t make sense.

I also tried to change it “manually” by editing the “public_html_dir” entry in the file /etc/webmin/virtual-server/domains/[random_number_series] for this sub-server website, but this change does not seem to become effective (or the path listed under Virtualmin > Server Configuration > Website Options – Website documents sub-directory is not the webroot path which serves as input for Let’s Encrypt Virtualmin SSL creation).

Ok, so maybe if no one could help me with changing the webroot path as identified by the Let’s Encrypt tab within Virtualmin for sub-servers… I think another way of solving it would be to know which exactly is the certbot command executed when the user clicks on “Request Certificate” through Let’s Encrypt tab.

The disadvantage of doing it this way is that I would not be able to automate the certificate renewal + would have to manually insert the new certificate in the apache vhost configuration. But at least I would be able to enable SSL for the website.