"lost connection after STARTTLS" error

OS: Ubuntu Linux 18.04.5, Webmin: 1.973, Virtualmin: 6.16, Usermin: 1.823, Authentic Theme: 19.73


Whenever I try to send emails from my webserver (called say, SERVER1) to an external SMTP server that I have set up (called say, SERVER2) I kept on getting the following error in SERVER2’s mail.logs:

postfix/smtpd[16991]: lost connection after STARTTLS from SERVER1[ipv6 address]
postfix/smtpd[16991]: disconnect from SERVER1[ipv6 address] ehlo=1 starttls=1 commands=2

What could be causing this issue?

Note:

  • In SERVER1, I have already copied a Let’s Encrypt SSL certificate from one of the virtual servers to Postfix (as well as to Dovecot and the other services).
  • I have a few other Virtualmin webservers that uses the SMTP server (SERVER2) and everything works fine for them.
  • saslauthd service is running.

SERVER1 - main.cf

smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu)
biff = no

# appending .domain is the MUA's job.
append_dot_mydomain = no

# Uncomment the next line to generate "delayed mail" warnings
#delay_warning_time = 4h

readme_directory = no

# See http://www.postfix.org/COMPATIBILITY_README.html -- default to 2 on
# fresh installs.
compatibility_level = 2

# TLS parameters
smtpd_tls_cert_file = /etc/postfix/postfix.cert.pem
smtpd_tls_key_file = /etc/postfix/postfix.key.pem
smtpd_use_tls=yes
smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache

# See /usr/share/doc/postfix/TLS_README.gz in the postfix-doc package for
# information on enabling SSL in the smtp client.

smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated defer_unauth_destination
myhostname = %SERVER1 HOSTNAME%
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
myorigin = /etc/mailname
mydestination = $myhostname, localhost.localdomain, %SERVER1 HOSTNAME%, localhost.%SERVER1 DOMAIN NAME%, localhost
mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128
mailbox_command = /usr/bin/procmail-wrapper -o -a $DOMAIN -d $LOGNAME
mailbox_size_limit = 0
recipient_delimiter = +
inet_interfaces = all
inet_protocols = all
virtual_alias_maps = hash:/etc/postfix/virtual
sender_bcc_maps = hash:/etc/postfix/bcc
sender_dependent_default_transport_maps = hash:/etc/postfix/dependent
home_mailbox = Maildir/
smtpd_sasl_auth_enable = yes
smtpd_tls_security_level = may
broken_sasl_auth_clients = yes
smtpd_recipient_restrictions = permit_mynetworks permit_sasl_authenticated reject_unauth_destination
smtp_tls_security_level = dane
allow_percent_hack = no
smtpd_tls_CAfile = /etc/postfix/postfix.ca.pem
smtpd_tls_mandatory_protocols = !SSLv2, !SSLv3, !TLSv1, !TLSv1.1
milter_default_action = accept
milter_protocol = 2
smtpd_milters = inet:localhost:8891
non_smtpd_milters = inet:localhost:8891
smtp_dns_support_level = dnssec
smtp_host_lookup = dns

SERVER1 - master.cf

# ==========================================================================
# service type  private unpriv  chroot  wakeup  maxproc command + args
#               (yes)   (yes)   (no)    (never) (100)
# ==========================================================================
smtp	inet	n	-	y	-	-	smtpd -o smtpd_sasl_auth_enable=yes -o smtpd_tls_security_level=may
#smtp      inet  n       -       y       -       1       postscreen
#smtpd     pass  -       -       y       -       -       smtpd
#dnsblog   unix  -       -       y       -       0       dnsblog
#tlsproxy  unix  -       -       y       -       0       tlsproxy
#submission inet n       -       y       -       -       smtpd
#  -o syslog_name=postfix/submission
#  -o smtpd_tls_security_level=encrypt
#  -o smtpd_sasl_auth_enable=yes
#  -o smtpd_tls_auth_only=yes
#  -o smtpd_reject_unlisted_recipient=no
#  -o smtpd_client_restrictions=$mua_client_restrictions
#  -o smtpd_helo_restrictions=$mua_helo_restrictions
#  -o smtpd_sender_restrictions=$mua_sender_restrictions
#  -o smtpd_recipient_restrictions=
#  -o smtpd_relay_restrictions=permit_sasl_authenticated,reject
#  -o milter_macro_daemon_name=ORIGINATING
#smtps	inet	n	-	y	-	-	smtpd -o syslog_name=postfix/smtps -o smtpd_tls_wrappermode=yes -o smtpd_sasl_auth_enable=yes -o smtpd_reject_unlisted_recipient=no -o smtpd_client_restrictions=$mua_client_restrictions -o smtpd_helo_restrictions=$mua_helo_restrictions -o smtpd_sender_restrictions=$mua_sender_restrictions -o smtpd_recipient_restrictions= -o smtpd_relay_restrictions=permit_sasl_authenticated,reject -o milter_macro_daemon_name=ORIGINATING
#628       inet  n       -       y       -       -       qmqpd
pickup    unix  n       -       y       60      1       pickup
cleanup   unix  n       -       y       -       0       cleanup
qmgr      unix  n       -       n       300     1       qmgr
#qmgr     unix  n       -       n       300     1       oqmgr
tlsmgr    unix  -       -       y       1000?   1       tlsmgr
rewrite   unix  -       -       y       -       -       trivial-rewrite
bounce    unix  -       -       y       -       0       bounce
defer     unix  -       -       y       -       0       bounce
trace     unix  -       -       y       -       0       bounce
verify    unix  -       -       y       -       1       verify
flush     unix  n       -       y       1000?   0       flush
proxymap  unix  -       -       n       -       -       proxymap
proxywrite unix -       -       n       -       1       proxymap
smtp      unix  -       -       y       -       -       smtp
relay     unix  -       -       y       -       -       smtp
        -o syslog_name=postfix/$service_name
#       -o smtp_helo_timeout=5 -o smtp_connect_timeout=5
showq     unix  n       -       y       -       -       showq
error     unix  -       -       y       -       -       error
retry     unix  -       -       y       -       -       error
discard   unix  -       -       y       -       -       discard
local     unix  -       n       n       -       -       local
virtual   unix  -       n       n       -       -       virtual
lmtp      unix  -       -       y       -       -       lmtp
anvil     unix  -       -       y       -       1       anvil
scache    unix  -       -       y       -       1       scache
#
# ====================================================================
# Interfaces to non-Postfix software. Be sure to examine the manual
# pages of the non-Postfix software to find out what options it wants.
#
# Many of the following services use the Postfix pipe(8) delivery
# agent.  See the pipe(8) man page for information about ${recipient}
# and other message envelope options.
# ====================================================================
#
# maildrop. See the Postfix MAILDROP_README file for details.
# Also specify in main.cf: maildrop_destination_recipient_limit=1
#
maildrop  unix  -       n       n       -       -       pipe
  flags=DRhu user=vmail argv=/usr/bin/maildrop -d ${recipient}
#
# ====================================================================
#
# Recent Cyrus versions can use the existing "lmtp" master.cf entry.
#
# Specify in cyrus.conf:
#   lmtp    cmd="lmtpd -a" listen="localhost:lmtp" proto=tcp4
#
# Specify in main.cf one or more of the following:
#  mailbox_transport = lmtp:inet:localhost
#  virtual_transport = lmtp:inet:localhost
#
# ====================================================================
#
# Cyrus 2.1.5 (Amos Gouaux)
# Also specify in main.cf: cyrus_destination_recipient_limit=1
#
#cyrus     unix  -       n       n       -       -       pipe
#  user=cyrus argv=/cyrus/bin/deliver -e -r ${sender} -m ${extension} ${user}
#
# ====================================================================
# Old example of delivery via Cyrus.
#
#old-cyrus unix  -       n       n       -       -       pipe
#  flags=R user=cyrus argv=/cyrus/bin/deliver -e -m ${extension} ${user}
#
# ====================================================================
#
# See the Postfix UUCP_README file for configuration details.
#
uucp      unix  -       n       n       -       -       pipe
  flags=Fqhu user=uucp argv=uux -r -n -z -a$sender - $nexthop!rmail ($recipient)
#
# Other external delivery methods.
#
ifmail    unix  -       n       n       -       -       pipe
  flags=F user=ftn argv=/usr/lib/ifmail/ifmail -r $nexthop ($recipient)
bsmtp     unix  -       n       n       -       -       pipe
  flags=Fq. user=bsmtp argv=/usr/lib/bsmtp/bsmtp -t$nexthop -f$sender $recipient
scalemail-backend unix	-	n	n	-	2	pipe
  flags=R user=scalemail argv=/usr/lib/scalemail/bin/scalemail-store ${nexthop} ${user} ${extension}
mailman   unix  -       n       n       -       -       pipe
  flags=FR user=list argv=/usr/lib/mailman/bin/postfix-to-mailman.py
  ${nexthop} ${user}

submission	inet	n	-	y	-	-	smtpd -o smtpd_sasl_auth_enable=yes -o smtpd_tls_security_level=may
smtps	inet	n	-	y	-	-	smtpd -o smtpd_sasl_auth_enable=yes -o smtpd_tls_security_level=may -o smtpd_tls_wrappermode=yes

Forgive me, I may have missed something. If you’re using SERVER2 as your SMTP server, shouldn’t Postfix on SERVER1 be setup to deliver emails indirectly via SERVER2?

/etc/postfix/main.cf

# specify SMTP relay host
relayhost = [SERVER2]:587

The only other thing that occurred to me was that possibly the certificate that SERVER1 is presenting doesn’t match the host email address or whatever domain is indicated in the email’s header. Postfix has a way of handling virtual hosts that I’ve never quite understood.

I attempted to run my own email servers for a few years, but ultimately decided they were more trouble than they were worth. Postfix was easily the #1 target on my servers for hackers, it received a non-stop flood of auth requests, and Fail2Ban was largely powerless to slow them down. I considered going with a self-hosted solution that was designed specifically for email (https://mailinabox.email) but in the end I took the easier route and purchased a multi-tenant mail service from MXRoute. It only costs about $40/year for the amount of storage I need, and I can still offer my clients hosted email services with custom domains. I just set up the necessary DNS and autoconfig.xml settings in Virtualmin’s template so provisioning is still fairly automated. But from context it sounds like you may be depending on the sendmail function on SERVER1 (or some other kind of email delivery that doesn’t involve configuring application-specific SMTP settings).

Hope you get your issue resolved!

That would be one way of sending mail via an external server but that didn’t work either.

Another way included setting up system emails from Webmin to be sent via an external SMTP.

And yet another way to use SMTP would be to use PHP on one of the sites in the server to send via an external SMTP (e.g. modifying wp_mail or using an SMTP plugin in WordPress sites).

None of the above works and I think it might have something to do with Postfix though I’m not sure yet.

do you use ivp6 on your server?

Yes I do have a shared ipv6 address for all the virtual servers on the server.

well ivp6 is great but it is also bad due complicated translation of the resources, just turn it off as never existed, you should be fine.

Setting this to ‘no’ should be a temp-fix at least.
I’ve encountered this in the past, but I don’t recall the proper solution as of know.
When relaying to server2, are you using hostname or IP?
Also, what does the logs on server2 say regarding the connection?

@toreskev I’m using the hostname and SERVER2 returns the following whenever I try to relay mail from SERVER1:

postfix/smtpd[16991]: lost connection after STARTTLS from SERVER1[ipv6 address]
postfix/smtpd[16991]: disconnect from SERVER1[ipv6 address] ehlo=1 starttls=1 commands=2

Even though the postfix files (main.cf and master.cf) are exactly the same (excluding server exclusive details like hostname) in SERVER1 and the other servers that can connect to SERVER2 successfully, I noticed one difference between the faulty SERVER1 and the other working servers though.

When I run netstat -anpe | grep "587" | grep "LISTEN" to check what is listening to port 587, the working servers return the following:

tcp        0      0 0.0.0.0:587             0.0.0.0:*               LISTEN      0          2552522308 15763/master
tcp6       0      0 :::587                  :::*                    LISTEN      0          2552522309 15763/master
unix  2      [ ACC ]     STREAM     LISTENING     4099958744 270/saslauthd        /var/spool/postfix/var/run/saslauthd/mux

Whereas the faulty SERVER1 returns the following:

tcp        0      0 0.0.0.0:587             0.0.0.0:*               LISTEN      0          3861966547 7910/master
tcp6       0      0 :::587                  :::*                    LISTEN      0          3861966548 7910/master

It seems like saslauthd is not listening to port 587 unlike the other working servers. What could be the reason for this since saslauthd seems to be active and running as seen below:

● saslauthd.service - LSB: saslauthd startup script
   Loaded: loaded (/etc/init.d/saslauthd; generated)
   Active: active (running) since Tue 2021-04-06 07:49:57 UTC; 1 weeks 3 days ago
     Docs: man:systemd-sysv-generator(8)
    Tasks: 2 (limit: 4915)
   CGroup: /system.slice/saslauthd.service
           ├─208 /usr/sbin/saslauthd -a pam -c -m /var/spool/postfix/var/run/saslauthd -r -n 2
           └─209 /usr/sbin/saslauthd -a pam -c -m /var/spool/postfix/var/run/saslauthd -r -n 2

I think the issue is related to ipv6 as you stated because when I try to send a test mail from webmin just now, I get the following error:

sending failed : Failed to IPv6 connect to [SERVER2]:587 : Connection timed out

But I have ipv6 on the other working servers as well though and they are working fine so I’m not sure what exactly is causing the ipv6/saslauthd related issue on this particular one?

RESOLVED:

SASL needs plain text authentication and the problem was with the “noplaintext” in my main.cf configuration. Since I am using TLS encryption, allowing plain text hopefully shouldn’t be a security issue.

(Still not sure how the other servers work fine though. Will fiddle with Postfix a lil bit more and see if there’s something I must have missed.)