Let's Encrypt won't follow redirect to URL with port 10000

Hmmm. With reference to your comment from the other thread which I have quoted below, in System Settings → Server Templates → Apache website could you set URL for admin redirect to use domain name. The custom URL you have applied there could be the cause of all your troubles.