Usermin cannot send emails unless smtpd_tls_security_level is set to 'may', then locally delivered emails are greylisted

I noticed last night that Usermin was unable to send emails to local or external addresses. Attempting to send an email returns this error in Usermin:

Failed to send mail : SMTP command mail from: <user@domain> failed : 530 5.7.0 Must issue a STARTTLS command first

I read an old forum thread which mentioned changing smtpd_tls_security_level from encrypt to may. Before doing that I attempted to enable “Use SSL for SMTP connection?” in the “Read Mail” module Usermin configuration, no success.

Setting the above smtpd variable from encrypt to may results in Usermin being able to talk to Postfix, however the send attempt is then subjected to greylisting. Usermin just reports this error and doesn’t send the email, so you have to wait and then reattempt the send after the timer has elapsed.

This is odd, because I wouldn’t expect outbound emails to be greylisted (seems they all are!). Also, emails from one user to a second user on the same physical server are being greylisted, which is going to prove inconvenient.

As this server is a migration from an old machine. The two are different control panel and software versions, so I manually updated the main.cf to incorporate much of the old security settings (mostly TLS tweaks) then some updates to take advantage of newer Postfix options.

Other webmail software (e.g. Rainloop) don’t have the STARTTLS option, and other clients are able to connect fine.

It’s my intention to force encryption, so setting smtpd_tls_security_level = may is out of the question. If Usermin doesn’t work with that setting, that’s OK, I have other webmail software I can deploy. However postgrey greylisting outbound emails, and emails between two local users, is a strange and unwanted behaviour.

I’m sure I’ve done something somewhere to cause this problem, but I can’t see the wood for the trees at the moment. I’d appreciate a second opinion.

postconf -n
(note, in the actual config the “hostname,anonymised.tld” line is the proper FQDN with matching PTR. I explicitly declared it to see if it might help to solve the greylisting problem.)

alias_database = hash:/etc/aliases
alias_maps = hash:/etc/aliases
allow_percent_hack = no
bounce_queue_lifetime = 4d
broken_sasl_auth_clients = yes
command_directory = /usr/sbin
config_directory = /etc/postfix
daemon_directory = /usr/libexec/postfix
data_directory = /var/lib/postfix
debug_peer_level = 2
debugger_command = PATH=/bin:/usr/bin:/usr/local/bin:/usr/X11R6/bin ddd $daemon_directory/$process_name $process_id & sleep 5
disable_vrfy_command = yes
header_checks = pcre:/etc/postfix/header_checks.regexp
home_mailbox = Maildir/
html_directory = no
inet_protocols = all
mailbox_command = /usr/bin/procmail-wrapper -o -a $DOMAIN -d $LOGNAME
mailbox_size_limit = 0
mailq_path = /usr/bin/mailq.postfix
manpage_directory = /usr/share/man
maximal_queue_lifetime = 4d
message_size_limit = 31457280
milter_default_action = accept
milter_protocol = 2
mydestination = $myhostname, localhost.$mydomain, localhost, $mydomain, hostname.anonymised.tld
newaliases_path = /usr/bin/newaliases.postfix
non_smtpd_milters = local:/run/milter-greylist/milter-greylist.sock,inet:localhost:8891
queue_run_delay = 300s
readme_directory = /usr/share/doc/postfix-2.10.1/README_FILES
sample_directory = /usr/share/doc/postfix-2.10.1/samples
sender_bcc_maps = hash:/etc/postfix/bcc
sender_dependent_default_transport_maps = hash:/etc/postfix/dependent
sendmail_path = /usr/sbin/sendmail.postfix
setgid_group = postdrop
smtp_tls_CAfile = /etc/ssl/certs/ca-bundle.crt
smtp_tls_ciphers = high
smtp_tls_exclude_ciphers = ${smtpd_tls_exclude_ciphers}
smtp_tls_loglevel = 2
smtp_tls_mandatory_ciphers = high
smtp_tls_mandatory_exclude_ciphers = MD5, aDSS, kECDH, kDH, SEED, IDEA, DES, ADH, RC2, RC4, RC5, PSD, SRP, 3DES, eNULL, aNULL
smtp_tls_security_level = may
smtp_use_tls = yes
smtpd_client_restrictions = check_client_access cidr:/etc/postfix/client_checks.cidr, reject_rhsbl_client dbl.spamhaus.org, reject_rbl_client sbl.spamhaus.org
smtpd_data_restrictions = reject_unauth_pipelining
smtpd_helo_required = yes
smtpd_milters = local:/run/milter-greylist/milter-greylist.sock,inet:localhost:8891
smtpd_recipient_limit = 100
smtpd_recipient_restrictions = check_policy_service unix:/var/spool/postfix/postgrey/socket
smtpd_relay_restrictions = permit_mynetworks, permit_sasl_authenticated, defer_unauth_destination, reject_unauth_destination, reject_non_fqdn_recipient, reject_unknown_reverse_client_hostname, permit_inet_interfaces
smtpd_sasl_auth_enable = yes
smtpd_sender_restrictions = check_sender_access pcre:/etc/postfix/sender_restrictions_bad_tlds.regexp
smtpd_tls_CAfile = /etc/postfix/postfix.ca.pem
smtpd_tls_auth_only = yes
smtpd_tls_cert_file = /etc/postfix/postfix.cert.pem
smtpd_tls_ciphers = high
smtpd_tls_dh1024_param_file = /etc/postfix/dh2048.pem
smtpd_tls_exclude_ciphers = aNULL, eNULL, EXPORT, DES, RC4, MD5, PSK, aECDH, EDH-DSS-DES-CBC3-SHA, EDH-RSA-DES-CBC3-SHA, KRB5-DES, CBC3-SHA
smtpd_tls_key_file = /etc/postfix/postfix.key.pem
smtpd_tls_loglevel = 2
smtpd_tls_mandatory_ciphers = high
smtpd_tls_mandatory_exclude_ciphers = MD5, aDSS, kECDH, kDH, SEED, IDEA, DES, ADH, RC2, RC4, RC5, PSD, SRP, 3DES, eNULL, aNULL
smtpd_tls_mandatory_protocols = !SSLv2, !SSLv3, !TLSv1, !TLSv1.1
smtpd_tls_received_header = yes
smtpd_tls_security_level = encrypt
smtpd_use_tls = yes
strict_rfc821_envelopes = yes
tls_high_cipherlist = EDH+CAMELLIA:EDH+aRSA:EECDH+aRSA+AESGCM:EECDH+aRSA+SHA256:EECDH:+CAMELLIA128:+AES128:+SSLv3:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!DSS:!RC4:!SEED:!IDEA:!ECDSA:kEDH:CAMELLIA128-SHA:AES128-SHA
tls_random_source = dev:/dev/urandom
unknown_address_reject_code = 554
unknown_client_reject_code = 554
unknown_hostname_reject_code = 554
unknown_local_recipient_reject_code = 550
virtual_alias_maps = hash:/etc/postfix/virtual

master.cf:

smtp    inet    n       -       n       -       -       smtpd -o smtpd_sasl_auth_enable=yes 
smtps   inet    n       -       n       -       -       smtpd
   -o syslog_name=postfix/smtps
   -o smtpd_tls_wrappermode=yes
   -o smtpd_sasl_auth_enable=yes
   -o smtpd_reject_unlisted_recipient=no
#   -o smtpd_client_restrictions=$mua_client_restrictions
#   -o smtpd_helo_restrictions=$mua_helo_restrictions
#   -o smtpd_sender_restrictions=$mua_sender_restrictions
#   -o smtpd_recipient_restrictions=permit_sasl_authenticated,reject
   -o milter_macro_daemon_name=ORIGINATING

I commented out four lines - the three $mua_ variables are not declared anywhere, and were just logging warning/errors in the logfile. The smtpd_recipient_restriction line seemed to be overriding the one in main.cf and was on the verge of giving me a severe migraine at 1am (server 550 rejecting all external emails!)

milter-greylist config:

socket "/run/milter-greylist/milter-greylist.sock" 666
dumpfile "/var/lib/milter-greylist/db/greylist.db" 600
geoipdb "/usr/share/GeoIP/GeoIP.dat"
dumpfreq 1
user "grmilter"


# Log milter-greylist activity to a file
#stat ">>/var/milter-greylist/greylist.log" \
stat ">>/var/log/milter-greylist.log" \
      "%T{%Y/%m/%d %T} %d [%i] %f -> %r %S (ACL %A) %Xc %Xe %Xm %Xh\n"
# Same, sent to syslog
#stat "|logger -p local7.info" \
#      "%T{%Y/%m/%d %T} %d [%i] %f -> %r %S (ACL %A) %Xc %Xe %Xm %Xh"

# Be verbose (or use -v flag)
#verbose

# Do not tell spammer how long they have to wait
#quiet

# MX peering
#peer 192.0.2.17
#peer 192.0.2.18

# Your own network, which should not suffer greylisting
list "my network" addr { 127.0.0.1/8 10.0.0.0/8 192.0.2.0/24 }

list "broken mta" addr {   \
        12.5.136.141/32    \ # Southwest Airlines (unique sender)
(...etc)
}

# List of users that want greylisting
list "grey users" rcpt {  \
        user1@example.com \
        user2@example.com \
        user3@example.com \
}

# Give this a try if you enabled DNSRBL
#dnsrbl "SORBS DUN" dnsbl.sorbs.net 127.0.0.10
#dnsrbl "SBL" sbl-xbl.spamhaus.org 127.0.0.2
#dnsrbl "CBL" sbl-xbl.spamhaus.org 127.0.0.4
#dnsrbl "NJABL" sbl-xbl.spamhaus.org 127.0.0.5
#dnsrbl "PBL" zen.spamhaus.org 127.0.0.10/31
#dnsrbl "MTAWL" list.dnswl.org 127.0.0.0/16

# Here is an example of user preference pulled from a LDAP directory
# (requires building --with-libcurl). If the milterGreylistStatus
# attribute is set to TRUE, then $usrRBL will be usable later in the
# ACL and will carry the values of the usrRBL attribute.
# urlcheck "userpref" \
# "ldap://localhost/dc=example,dc=net?milterGreylistStatus,usrRBL?one?mail=%r" \
# 30 getprop clear fork

racl continue from /.*/ addheader "X-Greylist: inspected by %V for IP:'%i' DOMAIN:'%d' HELO:'%h' FROM:'%f' RCPT:'%r'"

# And here is the access list
racl whitelist list "my network"
racl whitelist list "broken mta"
#racl whitelist dnsrbl "MTAWL"
#racl blacklist urlcheck "userpref" $usrRBL "CBL" dnsrbl "CBL" \
#               msg "Sender IP caught in CBL blacklist"
#racl blacklist $usrRBL "SBL" dnsrbl "BBL" \
#               msg "Sender IP caught in SBL blacklist"
#racl blacklist $usrRBL "NJABL" dnsrbl "NJABL" \
#               msg "Sender IP caught in NJABL blacklist"
#racl greylist list "grey users" dnsrbl "SORBS DUN" delay 24h autowhite 3d
racl greylist list "grey users" delay 30m autowhite 3d
ratelimit "domain_157808777223121" rcpt 1000 / 1h
racl blacklist from /.*@hosted-domain1.com/ ratelimit "domain_157808777223121" msg "Message quota exceeded"
racl whitelist from /.*@hosted-domain1.com/
ratelimit "domain_157817009622492" rcpt 1000 / 1h
racl blacklist from /.*@hosted-domain2.com/ ratelimit "domain_157817009622492" msg "Message quota exceeded"
racl whitelist from /.*@hosted-domain2.com/
racl whitelist default

hosted-domain1.com” and “hosted-domain2.com” are anonymised, in the config those are actual domains on the system.

Only a handful of domains on the system have per-account outbound rate limiting set up, partly for testing reasons. I don’t want inadvertent high volume email sending (whether compromised password or an ad-hoc mailshot) to knock out email service for the other domains.

I’ve also added the server’s FQDN and all locally hosted domains to /etc/postfix/postgrey_whitelist_clients.local - they weren’t originally, though adding them and restarting postgrey hasn’t solved it.

I’d appreciate any help with this, I’m not sure what the root cause is. (It’s probably me, but no sarcastic responses please :rofl:)

So, some progress on this. I subsequently realised milter-greylist is basically doing nothing (it’s an old version, plus it needs a manual upgrade to support the latest geolite2 databases). I think greypost was receiving the outbound emails because of an overly restrictive smtpd_recipient_restrictions line (smtpd_recipient_restrictions = check_policy_service unix:/var/spool/postfix/postgrey/socket).

I definitely have more reading to do regarding the order of precedence concerning smtpd_recipient_restrictions and smtpd_relay_restrictions.

I’ve altered smtpd_recipient_restrictions to be
smtpd_recipient_restrictions = permit_mynetworks permit_sasl_authenticated reject_unauth_destination check_policy_service unix:/var/spool/postfix/postgrey/socket permit_mx_backup permit_inet_interfaces reject_unknown_reverse_cli ent_hostname

smtpd_relay_restrictions is now
smtpd_relay_restrictions = permit_mynetworks, permit_sasl_authenticated, defer_unauth_destination, reject_non_fqdn_recipient, reject_unknown_reverse_client_hostname, permit_inet_interfaces

I’ve also added the server’s HELO, 127.0.0.1 and all outbound domains (for good measure) to postgrey_whitelist_clients.local. Perhaps this was unnecessary given I’ve modified the smtpd_ postfix config lines. Postfix now seems to now be correctly whitelisting outbound email.

I’ve also realised that I was being too BOFH by defining smtpd_tls_security_level as encrypt; per RFC 2487, I should permit unencrypted incoming connections, so it’s now set to may.

What was interesting was that, having defined only TLS1.2 as a supported protocol, I noticed a few third-party MTAs attempting to negotiate encrypted channels to deliver email with wildly obsolete TLS protocols and versions, so I’ve let them know. That was useful, at least. I was amused that this only revealed itself once I decided to finally deprecate support for TLS1.0 and 1.1 on my side.

Yet to solve is Usermin not correctly talking STARTTLS over SMTP. (Should that work?)