Virtual Servers moved to new machine have problems - cannot get lets encrypt certs

[palomnik]

Operating system Ubuntu Linux 20.04.4
Webmin version 1.994 Usermin version 1.840
Virtualmin version 7.1-1

I am moving about 15 virtual servers from an instance of Virtualmin on Ubuntu 16.x to a newer server running 20.04 After restoring the archives from backups (as created by virtualmin), I still have to manually change the IPs in the relevant nginx.conf for that particular server. The website either has a bad gateway warning, or the cert doesn’t work, and if I try to get a new cert I get:

Requesting a certificate for piratesatoll.com, www.piratesatoll.com from Let's Encrypt ..
.. request failed : Web-based validation failed :
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator webroot, Installer None
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for piratesatoll.com
http-01 challenge for www.piratesatoll.com
Using the webroot path /home/piratesatoll/public_html for all unmatched domains.
Waiting for verification...
Challenge failed for domain piratesatoll.com
http-01 challenge for piratesatoll.com
Cleaning up challenges
Some challenges have failed.
IMPORTANT NOTES:
 - The following errors were reported by the server:

   Domain: piratesatoll.com
   Type:   unauthorized
   Detail: 2604:180:3:5a::83c5: Invalid response from
   http://piratesatoll.com/.well-known/acme-challenge/MYrB2hxpcMlIwmHqOA_dV3d9aPKT-wibIx16uKvSJw4:
   404

   To fix these errors, please make sure that your domain name was
   entered correctly and the DNS A/AAAA record(s) for that domain
   contain(s) the right IP address.

The /var/log/letsencrypt log does not have additional useful info. The DNS appears to be correct.

If I just create a new virtual server on the new server, and restore only the website from a duplicator pro archive, then everything works. The IPs in the relevant nginx.conf are correct, but I have to add the location code that makes wordpress work (I wish it were included in the skel). Then I can generate new certs and everything works.

I was hoping that moving a virtual server via backup/restore would “just work”, but it doesn’t seem to be the case.

Here is the info on the virtualmin instance that I am moving from:

Operating system Ubuntu Linux 16.04.7
Webmin version 1.994 Usermin version 1.840
Virtualmin version 7.1-1

[palomnik]

I should mention that a newly restored virtual server site either gives a bad gateway, a bad cert or simply skips to another website on the server.

[Randomz]

Firewall issue?

[palomnik]

I don’t know why the firewall would be an issue in one case and not in another.

Now, after the last site I restored, for some reason, any other site URL goes to that site and not the site that was working perfectly before.

[palomnik]

It appears that all URLs on the server are loading the carts for one particular site. I don’t know how this happened or how to undo it.

[Ilia]

Do you have correct IPs set in Apache config <VirtualHost> sections? Also, double check if you don’t have mixed configuration of IP and * in <VirtualHost>. It must be configured one way or another.

Moreover, you can try setting all <VirtualHost> directives to use <VirtualHost *:80> and <VirtualHost *:443>, restarting Apache and trying requesting SSL certificates once again.

[palomnik]

I am using Nginx and not Apache. I did see the Apache-based section for this problem in the docs (thank you for great docs), but there was not a section for Nginx.

[palomnik]

I found the problem. IP6 was misconfigured when I moved the first of the virtual servers over. I just had to edit the site-specific nginx conf files, and add the lines for listening on port 80 and 443 with the IP6. This seems strange - I would think that the IP4 info would be enough, and even then, it should not produce the site-skipping behavior.

I hope that the documentation will include more nginx examples in the future. I know that Apache is the main thrust, but as Nginx takes over server share, it will be more important.

[palomnik]

It turns out that the ip4 address in the nginx conf is enough, until you add one site with ip6, which then rules the world.

And this still does not solve the issue of not being able to restore older virtual servers from older machines. I’m thankful to have used Duplicator Pro as a secondary backup.