What ports need to be open for requesting a Lets Encrypt certificate?

I have my firewall set to block everything except ports:

  • 20000 (destination)
  • 10000 (destination)
  • 443 (destination)
  • 80 (destination)
  • 53 (destination & source)
  • 22 (destination)

However when the firewall is active requesting a Lets Encrypt certificate fails with a time-out downloading the certificate.

If I disable the firewall, it works.

What have I got configured wrong?

Outgoing traffic is allowed on all ports?