Let's Encrypt renewal errors with alias domains using parent webroot after 7.20.2

SYSTEM INFORMATION
OS type and version Rocky Linux 8.10
Webmin version 2.202
Virtualmin version 7.30.4
Webserver version Apache 2.4

I’m having trouble renewing Let’s Encrypt certificates since the recent changes to this area of Virtualmin (in think added in Virtualmin 7.20.2).

Renewal log looks like (identifying information redacted within stars, but all looked correct):

Renewing an existing certificate for *mydomain*.com

Certbot failed to authenticate some domains (authenticator: webroot). The Certificate Authority reported these problems:
  Domain: *mydomain*.com
  Type:   unauthorized
  Detail: *0000:0000:0000:0000:0000:0000:0000:0000*: Invalid response from https://www.*mydomain*.com/.well-known/acme-challenge/*XXXXXXXXXXXXXXXXX*/: 404

Hint: The Certificate Authority failed to download the temporary challenge files created by Certbot. Ensure that the listed domains serve their content from the provided --webroot-path/-w and that files created there can be downloaded from the internet.

Saving debug log to /var/log/letsencrypt/letsencrypt.log
Some challenges have failed.

Looking at letsencrypt.log all looks OK (aside from the failure), but this line suggests Virtualmin may be creating the challenge file in the wrong place:

2025-01-13 10:17:41,266:INFO:certbot._internal.auth_handler:Cleaning up challenges
2025-01-13 10:17:41,266:DEBUG:certbot._internal.plugins.webroot:Removing /home/*myaccount*/domains/*mydomain*.com/public_html/.well-known/acme-challenge/*XXXXXXXXXXXXXXXXX*

My setup is slightly unusual in that the mydomain.com is a Sub-server of myaccount but they share the same webroot as the all run off the same codebase, but with their own SSL.

I would therefore expect to see the challenge file stored in:

/home/*myaccount*/public_html/.well-known/acme-challenge/*XXXXXXXXXXXXXXXXX*

This was working correctly previously, although I can’t be sure which directory the challenge was stored in when it was working.

It would be useful to see the command passed to certbot in the virtualmin renewal/error log, so I can see what virtualmin is passing to --webroot-path.

I’m going to see if I can find a way to allow requests to .well-known to load from the /domains/* folders while continuing to serve website content from the parent folder.

Any help appreciated, as I have one live site down at the moment and more with certificates expiring soon.

As a temporary fix to get the site up again I switched this sub-server back to serving it’s content from the ~/domains/mydomain.com/public_html folder, successfully renewed the certificate and then switched the webroot back to the ~/public_html to get the site loading correctly.

I think what the behaviour should be, and was previously, is that Virtualmin should read the correct webroot from the DocumentRoot for the Virtrual Host that it’s renewing for and pass that to certbot’s --webroot-path option?

Any news on this? I’m still having to manually tweak things around every time a certificate needs renewing, and I have around 50 domains set up on the system so it’s a bit of a pain :slight_smile:

I’m having a hard time understanding how this could be a Virtualmin issue!?


I’d recommend checking out this tutorial:

public_html/.well-know/acme-challenge

I managed to renew one of the websites that also didn’t happen, changing the .htaccess in accordance with other websites that had already been made.

<IfModule mod_rewrite.c>
    RewriteEngine off
</IfModule>

# Require all granted
Satisfy Any
Order Deny,Allow
Allow from all

<FilesMatch "^\.">
    # Require all denied
    Order Allow,Deny
    Deny from all
</FilesMatch>

Hi Ilia,

I’m familiar with how Let’s Encrypt certificates work. And they do continue to work for accounts configured normally.

The issue I’m having is specifically where I’ve set up a sub-server which uses the parent servers DocumentRoot.

Before the recent changes to how Virtualmin renews certificates, Virtualmin appeared to recognise the different DocumentRoot and place the .well-known folder in the parent accounts public folder (e.g. /home/myaccount/public_html/.well-known) however after the recent update, it how places it in the sub-servers public folder (e.g. /home/myaccount/domains/mysubserverdomain.com/public_html/.well-known) which of course isn’t accessible from the web due to the change in DocumentRoot and therefore the renewal fails.

This was working recently and then stopped working, and I’m 99% sure the only change was a virtualmin upgrade.

Is there a way I can downgrade Virtualmin to confirm my suspicions?

I don’t think this is related to the issue I’m having specifically, as the verification files are being placed in a folder which isn’t accessible (and not due to .htaccess).

Virtualmin doesn’t do any of that.

I don’t think you should do that. Instead, use the URL in question (e.g., https://www.*mydomain*.com/.well-known/acme-challenge/) and place a small text file, like test.txt, with the content Hello World. Then, try opening it in a browser. If you can’t access it, Certbot won’t be able to either…

Hi Ilia,

I created a file in /home/myaccount/public_html/.well-known/acme-challenge/test.txt and it is accessible from www.mydomain.com/.well-known/acme-challenge/test.txt

The problem is that the .well-known/acme-challenge/* files are being created in /home/myaccount/domain/mydomain.com/public_html/.well-known/acme-challenge - which isn’t publicly accessible, because it’s not the DocumentRoot for the Apache VirtualHost.

Update on this:

If I change the ‘Website documents sub-directory’ under ‘Website Options’ on the main account, this is used by Certbot. For example if I change it from ‘public_html’ to ‘public_files’ the process will work using /home/myaccount/public_files/.well-known…

Unfortunately for the sub-servers, I can’t change the ‘Website documents sub-directory’ to be that of the parent server, as it won’t accept and absolute path, or a ‘…’

Failed to save website options : The website documents sub-directory cannot contain ..

So I suppose this may actually be a feature request (although I’m perplexed as to why this setup was working previously) to either:

  • Automatically detect the actually folder which is the document root, from Apache’s DocumentRoot
  • OR allow Website Options → Website documents sub-directory to be set to use the Parent accounts directory
  • OR manually specify the websites document root on the SSL request page

In any case, I think it would be really useful to show in the error log which directory was being used. I suspect it would help solve a lot of Let’s Encrypt issues without people having to bug you on the forums :slight_smile:

For anyone else with this issue, I’ve solved it for now by adding an Alias for each of the sub-servers in their VirtualHost directive which maps the .well-known folder to the sub-servers public folder location:

Alias /.well-known/acme-challenge/ /home/myaccount/domains/mydomain.com/public_html/.well-known/acme-challenge/
<Directory /home/myaccount/domains/mydomain.com/public_html/.well-known/acme-challenge>
    Options None
    Require all granted
    AllowOverride None
</Directory>

Note: This is only needed if you are using a different DocumentRoot to the standard one set by Virtualmin.

That shouldn’t be necessary for an Alias. Something is going wrong somewhere, but I don’t know what.

Hi Joe,

Let me know if you need any more info. I’m not sure how things work under the hood in Virtualmin but it would be good to get more visibility in the UI for Let’s Encrypt of where the verification files are being placed.

Aside from the issue in this thread, what would be really cool would be the option to create a ‘Alias with own SSL’ for a parent server. The only reason I’m using Sub-servers for this set up at all is to avoid having lots of Subject Alternate Names specified on one certificate (due to Let’s Encrypt limits, client privacy and to prevent one client changing their DNS and preventing renewal for all)…