Troubleshooting HTTP for Let's Encrypt

No, there aren’t any .htaccess files in the directory. So here’s something weird:

I can renew the Webmin SSL certificate just fine (Webmin > Webmin Configuration > SSL Encryption > Let’s Encrypt > Request Certificate), but it didn’t update the Usermin certificate (Webmin > Usermin Configuration > SSL Encryption > Current Certificate). I had to do that manually by selecting the Copy Certificate From Webmin button.

But…any of the other domains in Virtualmin redirect to that 404 error.

Weird, got me stumped. Under the VM Server Configuration → Website Directs I have a redirect to ssl that maybe set as well. but you shouldn’t get 404

I’ve got that as well for both the default Webmin site and the VM sites.

Thats different to mine, I have as destination https://yoursite.com/$1

That was put in by the server when I first set up the site and I haven’t changed it.

And to further muddy the water, since Webmin uses a subdomain, I decided to try and request a new cert for the main domain. It gives me this error:

Validating configuration for foo-bar[dot]com …
… errors were found, which will prevent Let’s Encrypt from issuing a certificate :

Apache website : An IPv6 DNS record with address dddd:cccc:cbbb::aaaa:d363 exists, but this virtual server does not have IPv6 enabled

I use Cloudflare, and that’s why I’m getting an IPv6 address, but if I don’t want to use IPv6 on the server.

mxtoolbox has nothing useful here.

If you can’t browse to .well-known on the server filesystem, obviously LE can’t validate. You have to fix that. When you proxy or redirect, you must exclude .well-known from that and allow it to be served from the filesystem.

Joe, there shouldn’t be any proxying or redirecting whatsoever. That’s where I’m having the problem.

After some further digging, I found that I can browse a new test.html file I placed in the .well-known folder of the main Webmin server.

When I tried to access the original test.html from my first post, which had different content, I was immediately redirected to the main Webmin server’s file that I had just created.

Since the content was different, then that was the only clue I had. Now just to figure out why the redirect is occurring.

Even if there is no htaccess file in that directory, the htaccess file the parent directory will apply recursively.

You should provide @stefan1959 an unqualified answer to his very valid question.

@Joe @stefan1959

There are no .htaccess files upstream either.

Have you tried turning off the cloudflare proxy while troubleshooting?
You mentioned that you are using cloudflare.
Turn off the proxy to show direct. Perhaps there is a conflict?

Good call, but I haven’t turned it on yet for that subdomain. Just DNS only with no proxy.

Did you manually enter this?

I did not. All of the new servers I create have that automatically in the configuration.

I would like to suggest going through the list below on a virtual server that is not proxy through Cloudflare.
By the looks of it, you have problems with a re-direction and IPv6 not properly set up.
This is going by the errors you mentioned.

This can be for a subserver.mydomain.com or mydomain.com.

In Virtualmin > Subserver.mydomain.com > Server Configuration > Change IP Address:
Make sure your IPv4 and IPv6 address is correct.

In Virtualmin > Subserver.mydomain.com > Server Configuration > Website Redirects
Remove all redirects.

In Virtualmin > Subserver.mydomain.com > Server Configuration > SSL Certificate
Install your Let’s Encrypt SSL Cert.

Let us know what happens.

Hy @cyberndt, I appreciate you looking at this. I’ll try and clarify:

The subdomain for Webmin, webmin.mydomain.com, is able to obtain a certificate without problem. It is not proxied through Cloudflare.

I’m not using IPv6 at all. Not sure why that message came up for me the other day, but that at least has gone away.

I went ahead and created a test.mydomain.com account that is not proxied through Cloudflare. Before I created it, I verified that it had the correct IPv4 address.

And right there during creation it is getting a 404 error to the .well-known folder.

I checked for redirects, and there weren’t any.

But, I learned the reason it receives that error is because it is redirecting to mydomain.com/.well-known/acme-challenge from test.mydomain.com/.well-known/acme-challenge. (Notice it went from the subdomain to the root domain.) I figured this out when I performed a “Test connectivity” check during the SSL creation again and then attempted to load the site directly.

That gets us closer to the root cause. I’ll look into it more tomorrow. If you have any ideas in the meantime, I’d love to hear them.

That doesn’t sound like a redirect. If it is simply serving the wrong site, it sounds like an Apache VirtualHost misconfiguration. Apache decides which site to serve based on an algorithm nobody understands, but if you have mixed * and IP VirtualHosts, Apache will serve something you didn’t expect. Similarly, if you have IPv6 config anywhere in the Apache VirtualHosts but not for every domain, you’ll get surprises.

Some things to look at are covered in our Web Troubleshooting FAQ: Website Troubleshooting – Virtualmin

There are all sorts of ways you can end up there. Restoring a domain that uses * into a server that is using IP based VirtualHosts, installing any of a number of web apps from system packages can do weird things to VirtualHosts (so don’t do that), IPv6 misconfiguration in Virtualmin or having it turned on and then disabling it later without cleaning up the configs that were created, etc.

Apache has bizarre, absolutely incomprehensible, virtual host selection rules. So…you have to take away all possible ways it can misinterpret your intentions because it will absolutely misinterpret you if there’s any way for it to.

2 Likes

This totally made me chuckle. Thanks! Taken to heart and I’ll resume troubleshooting tomorrow.

1 Like

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.