Unable to track the spammer

Developers
1

Hi,

I am fedup locating the user/script which is sending out mail from our server. i am using postfix as MTA. I am not able to locate who is sending out the emails or the scripts which is already compromised. I have scanned the whole server using clamscan and unable to find out any infected files.

[root@site]# postconfig -n -bash: postconfig: command not found [root@site postfix]# postconf -n alias_database = hash:/etc/aliases alias_maps = hash:/etc/aliases allow_percent_hack = no broken_sasl_auth_clients = yes command_directory = /usr/sbin config_directory = /etc/postfix daemon_directory = /usr/libexec/postfix data_directory = /var/lib/postfix debug_peer_level = 2 home_mailbox = Maildir/ html_directory = no inet_interfaces = all inet_protocols = all mail_owner = postfix mailbox_command = /usr/bin/procmail-wrapper -o -a $DOMAIN -d $LOGNAME mailbox_size_limit = 0 mailq_path = /usr/bin/mailq.postfix manpage_directory = /usr/share/man milter_default_action = accept milter_protocol = 2 mydestination = $myhostname, localhost.$mydomain, localhost, cl-t205-013cl.privatedns.com newaliases_path = /usr/bin/newaliases.postfix non_smtpd_milters = local:/var/run/milter-greylist/milter-greylist.sock queue_directory = /var/spool/postfix readme_directory = /usr/share/doc/postfix-2.6.6/README_FILES sample_directory = /usr/share/doc/postfix-2.6.6/samples sender_bcc_maps = hash:/etc/postfix/bcc sendmail_path = /usr/sbin/sendmail.postfix setgid_group = postdrop smtpd_milters = local:/var/run/milter-greylist/milter-greylist.sock smtpd_sasl_auth_enable = yes smtpd_sasl_security_options = noanonymous unknown_local_recipient_reject_code = 550 virtual_alias_maps = hash:/etc/postfix/virtual

Around 100s of mails are being sent out from the server. I was checking the log /var/log/maillog. Currently I have stopped the postfix service since the server is already added in most of the blacklists. I am not familiar with postfix and only with exim since I used it before. Following are the sample logs :

Feb 26 02:38:56 cl-t205-013cl postfix/smtp[7283]: 14DE3300741: to=, relay=mailscanner-pri.connect.com.fj[119.235.102.5]:25, delay=654, delays=650/0.45/3.7/0, dsn=4.0.0, status=deferred (host mailscanner-pri.connect.com.fj[119.235.102.5] refused to talk to me: 554-mailscanner-pri.connect.com.fj 554 Your access to this mail system has been rejected due to the sending MTA's poor reputation. If you believe that this failure is in error, please contact the intended recipient via alternate means.) Feb 26 02:38:56 cl-t205-013cl postfix/smtp[7232]: D355C300742: to=, relay=mailgate1.grindrod.com[196.216.172.111]:25, delay=657, delays=653/0.27/3.9/0, dsn=4.0.0, status=deferred (host mailgate1.grindrod.com[196.216.172.111] refused to talk to me: 554-mailgate1.grindrod.com 554 Your access to this mail system has been rejected due to the sending MTA's poor reputation. If you believe that this failure is in error, please contact the intended recipient via alternate means.) Feb 26 02:38:56 cl-t205-013cl postfix/smtp[7243]: 2C98830075C: to=, relay=mail02.bdi-online.de[139.1.144.9]:25, delay=619, delays=617/0.89/0.38/0, dsn=4.0.0, status=deferred (host mail02.bdi-online.de[139.1.144.9] refused to talk to me: 554-fraosmx003.os-srv.de 554 Your access to this mail system has been rejected due to the sending MTA's poor reputation. If you believe that this failure is in error, please contact the intended recipient via alternate means.) Feb 26 02:38:56 cl-t205-013cl postfix/smtp[7233]: D355C300742: to=, relay=spool.mail.gandi.net[217.70.184.6]:25, delay=657, delays=653/0.28/0.53/3.6, dsn=2.0.0, status=sent (250 2.0.0 Ok: queued as 1873A2A8065) Feb 26 02:38:56 cl-t205-013cl postfix/smtp[7280]: 66C02300409: to=, relay=smtp2.iitd.ernet.in[103.27.10.44]:25, delay=4343, delays=4339/0.44/2.8/1.2, dsn=4.7.1, status=deferred (host smtp2.iitd.ernet.in[103.27.10.44] said: 450 4.7.1 Client host rejected: cannot find your hostname, [*.*.*.*.] (in reply to RCPT TO command)) Feb 26 02:38:56 cl-t205-013cl postfix/smtp[7298]: 8F2C83005DD: host hostrelay01.logix.in[115.112.214.41] refused to talk to me: 554-delta.logix.in 554 Your access to this mail system has been rejected due to the sending MTA's poor reputation. If you believe that this failure is in error, please contact the intended recipient via alternate means. Feb 26 02:38:56 cl-t205-013cl postfix/smtp[7212]: 9F110300645: to=, relay=hostrelay03.logix.in[115.112.241.70]:25, delay=4101, delays=4097/0.77/3.5/0, dsn=4.0.0, status=deferred (host hostrelay03.logix.in[115.112.241.70] refused to talk to me: 554-delta.logix.in 554 Your access to this mail system has been rejected due to the sending MTA's poor reputation. If you believe that this failure is in error, please contact the intended recipient via alternate means.) Feb 26 02:38:56 cl-t205-013cl postfix/smtp[7217]: 8F2C83005DD: to=, relay=hostrelay01.logix.in[115.112.214.41]:25, delay=4471, delays=4467/1.3/3/0, dsn=4.0.0, status=deferred (host hostrelay01.logix.in[115.112.214.41] refused to talk to me: 554-delta.logix.in 554 Your access to this mail system has been rejected due to the sending MTA's poor reputation. If you believe that this failure is in error, please contact the intended recipient via alternate means.) Feb 26 02:38:56 cl-t205-013cl postfix/smtp[7296]: BF61F300696: to=, relay=in.mx2.mailhostbox.com[115.114.58.15]:25, delay=1287, delays=1283/1.9/2.3/0.29, dsn=4.7.1, status=deferred (host in.mx2.mailhostbox.com[115.114.58.15] said: 450-4.7.1 Client host rejected: cannot find your reverse hostname, [*.*.*.*.] 450 4.7.1 Please see http://support.mailhostbox.com/email-administrators-guide/error-codes for explanation of the problem. (in reply to RCPT TO command))

Can I locate the actual user sending out the emails ? I have checked whether the server is open relay in :
http://mxtoolbox.com/diagnostic.aspx

SMTP Open Relay 	OK - Not an open relay.

I want to know if it is possible to set only one user who can send out emails from the server and block all others ?

Thanks,

2

Firstly, please put all shell listings in [code][/code] tags, otherwise they become unreadable due to lost linebreaks and monospace font.

You say you used “clamscan” to find the malicious script… Do you mean you used the virus scanner that comes with Virtualmin? In that case I’m not surprised that it didn’t find anything, since it’s not “tuned” for web malware, but only for stuff that tries to infect Windows machines and so on. :slight_smile:

You might want to download the software “Linux Malware Detect” and run it against your /home directory. That’s specifically for malicious web scripts. It can also run automatically and periodically, which you should have it do.

Regarding Postfix… Provided the bad script uses SMTP connections to deliver its stuff, you can do several things. For one, remove the directive permit_mynetworks from the lines smtpd_*_restrictions in the file /etc/postfix/main.cf. That will force all deliveries, also local ones, to be authenticated.

Local websites will then be unable to send out mail without SMTP authentication. (Be careful about this, it might break email delivery from legitimate sites, but you can certainly do this while searching for the bad website.) If a bad script tries to, you’ll see this in the mail.log and auth.log. In turn, when the bad script does authenticates itself, you can see the username used in the mail.log.

You can also do watch -n 1 "netstat -tpn | grep :25" to catch all connections being made from/to port 25 of your system. The number in the last column is the process ID, which you can then match against ps aux | grep ID to find out which website makes the connections. This is, provided you’re using FCGI as PHP execution mode, only then will each site get its own process.

If the bad script puts the spam directly in the Postfix queue using local delivery, it’s gonna be more tricky. In that case, you need to closely watch the mail.log, and disable one website after the other, until the spam influx stops. Actually, this disable-one-by-one method you can use in any case if you can’t catch the culprit otherwise.

In all cases, you should check the process list ps aux for any processes that should not be there, especially running non-php processes under users, or files in the /tmp directory or so.

With those steps combined, you should be able to find out the responsible site. If you need advanced help with this stuff, I can offer you to do a screen sharing and Skype session.

3

Howdy,

Also, are you able to see a copy of any of the spam mails that are going out? They will typically contain information regarding how they were sent, and what account sent them.

You may not actually have any malware, it could just be an account that was broken into, or a web app with a security issue, or similar.

If you can share the email headers from one of the messages that was sent out, we can review that and help identify the source.

-Eric

4

Yes Eric has a good point there. If you’re not sure whether it’s a regular email user or a website, it could well be the former. In that case, you have information about the user both in the mail.log and in the mail itself. In the mail you’ll have a received header like so:

Received: from HELONAME (HOSTNAME[1.2.3.4]]) (Authenticated sender: sendinguser@domain.tld) by MAILSERVERHOST.tld (Postfix) with ESMTPSA id 8BB134482D for <recipient@domain.tld>; Wed, 26 Feb 2014 16:47:08 +0100 (CET)

5

BTW Linux Malware Detect is located at https://www.rfxn.com/projects/linux-malware-detect/

R-fx Networks has some great admin scripts – I have used them for many years.

6

This can be a big pain. As the guys above said, especially Locutus, Maldet scanning and removing ‘permit_mynetworks’ to force authentication is a good way to go to quickly tackle the problem. Another quick way to trace and shutdown this type of problem is to view the list of Virtualserver logs, eg: ‘/var/log/virtualmin’ and sort according to ‘Last modification time’. The offending Virtualserver will quickly give itself away.