Let's Encrypt won't follow redirect to URL with port 10000

Not quite. They went in the order 1, 3, 2. I only edited the directives because the automatic renewal failed.