Letsencrypt - again :(

SYSTEM INFORMATION
OS type and version Ubuntu Linux 20.04.6
Webmin version 2.105
Usermin version 2.005
Virtualmin version 7.8.2
Theme version 21.09.5
Package updates All installed packages are up to date

Regarding to this (un-answered) thread (Again issue with letsencrypt / certbot - #2 by suther) I like to give another try to get an answer. Especially to the question:

Even if the problem is solved, I’d like to know, what do you guys say about to upgrade certbot at virtualmin environments for an up-to-date version (>=1.26.0) by default… AND to have the posibillity to really use “Wildcard certificates” based on DNS-Entriy-Plugin?
BTW: Here might be a one, we could integrate in Virtualmin: certbot-dns-standalone 6

I’ve done the steps I’ve described in the linked thread above (means install certbot via snap) and got certbot version: 2.7.4

If I run letsencrypt via Virtualmin I got this:

But if I run it via CLI:

certbot -d mydomain-5hb.de --dry-run certonly

It work as expected

image

Questions

  1. Why does Virtualmin still have the outdated 0.40 Version bundled by default… or may I have a miss-configuration in my server setup?
  2. Is there a solid way to keep certbot up-to-date with the “recent” certbot versions in any way?
  3. How to fix the issue Virtualmin has with the current certbot version… (that error: unknown command "certonly", see 'snap help'. error)

Yes. wait until OS updates it in the package.

It (LE) works as is for most (if not all folk)

There is a good reason why wildcard certs are not a good thing to use. (Joe gave this in some earlier answer (I cannot remember where exactly, without the effort of doing a search here https://forum.virtualmin.com/t/lets-encrypt-wildcard-certificate/121146 and elsewhere)

If Web-based validation fails it is usually because your web server is not permitting the LE to place its temorary file in the .well-known so check that.

DNS validation for obtaining wildcard SSL certificates is fully supported by Virtualmin, considering that Virtualmin has control over the domain’s DNS zone. There’s no need for manual changes. That said, you don’t need to install the latest version of certbot using snap or do anything else, as Virtualmin handles the necessary processes internally.

Ok, thank you for the feedback. But in my case, it never worked (on non of my webmin/virtualmin servers)

Was it implemented in the early past ?
How can I check / adjust the setup to make it work?

I remember (regarding to my first thread - I’ve linked it above) it never was updated and sometimes work, and sometimes not. It was always a pain.

That was the reason I followed the instruction to remove the “package” of certbot and install it via snap, because I had no chance to get a newer version as 0.40 via the package manager.

Caused of this, I really wonder that it seems to work for all of you as expected.
So what to do? Should I re-try to install it via apt (I use UBUNTU) and see if I got a newer version?

Or what’s with the suggestion of @Ilia … if I understood in right way, I wouldn’t need a certbot package at all if the DNS would work as expected for letsencrypt.
To be honest, I would love to have it work with DNS.

It has been around for years.

I think you should just revert to stock certbot and check our Let’s Encrypt related tutorial:

https://www.virtualmin.com/documentation/web/ssl-le/

You do need certbot specifically to get wildcard SSL certificate and DNS validation work. It can be installed as easy as:

apt-get install certbot

However, it is worth pointing out that certbot package must be already pre-installed and configured correctly, if Virtualmin was installed in recommended way, i.e. using virtualmin-install.sh script.

Yes it was initially. Only during all the trouble I removed certbot apt-package.
I’ve tried, and here again is the trouble i mean:

apt only have certbot 0.40.0 available!

Certbot has supported DNS wildcard certificates since version 0.22.0!

No worries!

so this 0.40 version is perfectly fine for a virtualmin installation?
I can give a try and see if it still make trouble.

Only for my understanding… why is this certbot version is so incredible old.

Yes!

Ubuntu 20.04 itself is old, so certbot package is old as well. If you want newer software, use newer OS.

You probably should direct that question to the OS package managers. But I would guess that the answer to be “they don’t need to” as the current version works just fine or there are issues with the newer version.

On the subject of uninstalling then reinstalling any package I would proceed with caution. You never know what may have been reconfigured after the original install.
back to your original questions:

Virtualmin takes whatever your OS has from the package manager. As that is currently 0.40, that is what is installed. I would not install anything else (including snapd) unless fully aware of the possible issues.

Ok, triggered now the renew via GUI with this outdated 0.40.0 and got this error.
Now I remember, it was the initial error, why I switch from 0.40.0 from package manager to snap-version:

Traceback (most recent call last):
  File "/usr/share/webmin/webmin/acme_tiny.py", line 198, in <module>
    main(sys.argv[1:])
  File "/usr/share/webmin/webmin/acme_tiny.py", line 194, in main
    signed_crt = get_crt(args.account_key, args.csr, args.acme_dir, log=LOGGER, CA=args.ca, disable_check=args.disable_check, directory_url=args.directory_url, contact=args.contact)
  File "/usr/share/webmin/webmin/acme_tiny.py", line 149, in get_crt
    raise ValueError("Challenge did not pass for {0}: {1}".format(domain, authorization))
ValueError: Challenge did not pass for www.waescherei-bimbom.de: {'identifier': {'type': 'dns', 'value': 'www.waescherei-bimbom.de'}, 'status': 'invalid', 'expires': '2023-11-28T15:08:28Z', 'challenges': [{'type': 'http-01', 'status': 'invalid', 'error': {'type': 'urn:ietf:params:acme:error:unauthorized', 'detail': '12.34.56.78: Invalid response from http://www.waescherei-bimbom.de/.well-known/acme-challenge/UkXx9Ki9eBxn1jnrvyu-l8IJq0MY_EJNbVwsgHiYdvM: 404', 'status': 403}, 'url': 'https://acme-v02.api.letsencrypt.org/acme/chall-v3/12375978746/xPH_dg', 'token': 'UkXx9Ki9eBxn1jnrvyu-l8IJq0MY_EJNbVwsgHiYdvM', 'validationRecord': [{'url': 'http://www.waescherei-bimbom.de/.well-known/acme-challenge/UkXx9Ki9eBxn1jnrvyu-l8IJq0MY_EJNbVwsgHiYdvM', 'hostname': 'www.waescherei-bimbom.de', 'port': '80', 'addressesResolved': ['12.34.56.78'], 'addressUsed': '12.34.56.78'}], 'validated': '2023-11-21T15:15:57Z'}]}

And “yep”, know I remember that this was the issue which force me to upgrade certbot manually.
What might the issue be that it doesn’t work for me, but seems to work for each other with a recent Virtualmin Version. Don’t see where the issue might be.

acme_tiny.py doesn’t support DNS validation. You should install certbot package and make sure that Webmin can find it. If certbot package is installed but Webmin still uses acme_tiny.py you should manually configure the path to certbot command, like /usr/bin/certbot in Webmin ⇾ Webmin Configuration ⇾ Configuration: Let’s Encrypt configuration page.

Ok, didn’t know that there is a fallback.

I’ve done as you described, and error changed now… so we are a step forward :+1:t3:

Now I got this:

NGINX has an regExp to allow acme-challenge:

Oh :flushed: you are right.
I was so busy on daily work that I didn’t recognize that a new LTS of Ubuntu came out in AUG 2023.

Is there a recommended way to do such an major release upgrade to next LTS?

22.04 is such a big change from 20.04 I am not sure about recommended but personally I did this through backups and starting with a new box with the new install (it was quite a task!)

Virtualmin does not bundle anything. Your OS provides packages.

I see your point, but I would try a general upgrade first… as it would save a huge amount of time.
Only the whole server-setup and hardening will take at least a day.

If it doesn’t work I could go anyway with a fresh installation and recover from backup.

Any Idea how to fix the above letsencrypt error for 20.04 (as long as I didn’t have done the upgrade to 22.04)?

No. Mainly because I still see nothing wrong with the certbot 0.40.0 version as supplied (even on the odd 20.04 OS I still have to upgrade). I’m not even sure if 22.04 comes with certbot 2.7.4 without looking. Perhaps that needs clarification before starting the upgrade.