SENDMAIL! Receiving ok, not sending

Virtualmin
1

Ok, so I am held back from dropping my paid hosting entirely by the fact that my sendmail is not working correctly. I’ve been researching and tinkering with this problem for months now and it’s high time I asked for help. I’m including all the relevant information I can think of in this post and will be checking it often as I need to resolve this. Please help!

I am running Ubuntu 8.0.4 LTS with Webmin, Usermin, and Virtualmin and multiple domains.

Sendmail was working for sending mail out but not receiving mail. Trying to get incoming mail working, I installed procmail and spamassassin both. I also installed the Webmin module for relaying domains (not that I needed it, but I did anyway.) As a sidenote, I also install pear modules, including mail. Now I have the opposite problem. I am able to receive mail perfectly, but unable to send mail out.

In Webmin’s sendmail configuration, under “Local Domains Cw” I have:

[code:1]
localhost
junkroomserver
junkroomserver.hsd33.az.comcast.net
domain1.com
domain2.com
domain3.com
192.168.1.99[/code:1]

Under Webmin’s sendmail “Outgoing Domains CG” I have:

[code:1]
localhost
junkroomserver.hsd33.az.comcast.net
domain1.com
domain2.com
domain3.com[/code:1]

I have all blanks, nothing entered for domain masquerading. I have not setup any address mappings, domain routing, outgoing addresses, domain mapping, or relay domains… other than what VirtualMin did for me when I added the virtual server for each domain using Virtualmin.

Here is the relevant part of my sendmail.cf file:

[code:1]
Other dnl # Masquerading options
Feature FEATURE(always_add_domain')dnl Other MASQUERADE_AS(junkroomserver.hsd33.az.comcast.net’)dnl
Feature FEATURE(allmasquerade')dnl Feature FEATURE(masquerade_envelope’)dnl
Feature FEATURE(virtusertable') Feature FEATURE(genericstable’)
Feature FEATURE(mailertable') Feature FEATURE(domaintable’)
Other dnl # Default Mailer setup
Other MAILER_DEFINITIONS
Mailer MAILER(local')dnl Mailer MAILER(smtp’)dnl
Mailer MAILER(`procmail’)
[/code:1]

Here is my /etc/hosts file:

[code:1]
127.0.0.1 localhost
192.168.1.99 junkroomserver.hsd33.az.comcast.net junkroomserver

The following lines are desirable for IPv6 capable hosts

::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts[/code:1]

Here is the mail.log entry for a mail that was sent successfully:

Oct 29 05:14:34 junkroomserver sendmail[9505]: m9TBEXw6009505: from=www-data, size=1597, class=0, nrcpts=1, msgid=<200810291114.m9TBEXw6009505@junkroomserver.hsd33.az.comcast.net>, relay=www-data@localhost Oct 29 05:14:34 junkroomserver sm-mta[9506]: m9TBEYux009506: from=<www-data@junkroomserver.hsd33.az.comcast.net>, size=1802, class=0, nrcpts=1, msgid=<200810291114.m9TBEXw6009505@junkroomserver.hsd33.az.comcast.net>, proto=ESMTP, daemon=MSP-v4, relay=localhost [127.0.0.1] Oct 29 05:14:34 junkroomserver sendmail[9505]: m9TBEXw6009505: to=vyoubaby@gmail.com, ctladdr=www-data (33/33), delay=00:00:01, xdelay=00:00:00, mailer=relay, pri=31597, relay=[127.0.0.1] [127.0.0.1], dsn=2.0.0, stat=Sent (m9TBEYux009506 Message accepted for delivery) Oct 29 05:14:34 junkroomserver sendmail[9510]: m9TBEYQU009510: from=www-data, size=575, class=0, nrcpts=1, msgid=<200810291114.m9TBEYQU009510@junkroomserver.hsd33.az.comcast.net>, relay=www-data@localhost

and here is a log entry from mail which was not sent successfully:

Oct 30 04:00:04 junkroomserver sm-msp-queue[7445]: m9TFiveL005813: to=vyoubaby@gmail.com, delay=18:15:07, xdelay=00:00:00, mailer=relay, pri=5880463, relay=[127.0.0.1] [127.0.0.1], dsn=4.0.0, stat=Deferred: Connection refused by [127.0.0.1] Oct 30 04:00:04 junkroomserver sm-msp-queue[7445]: m9TFacr8005360: to=vyoubaby@gmail.com, delay=18:23:26, xdelay=00:00:00, mailer=relay, pri=5970462, relay=[127.0.0.1] [127.0.0.1], dsn=4.0.0, stat=Deferred: Connection refused by [127.0.0.1] Oct 30 04:00:04 junkroomserver sm-msp-queue[7445]: m9TFDjCx006527: to=vyoubaby@gmail.com, delay=18:46:19, xdelay=00:00:00, mailer=relay, pri=6240460, relay=[127.0.0.1] [127.0.0.1], dsn=4.0.0, stat=Deferred: Connection refused by [127.0.0.1] Oct 30 04:00:04 junkroomserver sm-msp-queue[7445]: m9TF5ges006110: to=polybaby@polybaby.com, ctladdr=www-data (33/33), delay=18:54:22, xdelay=00:00:00, mailer=relay, pri=6243299, relay=[127.0.0.1] [127.0.0.1], dsn=4.0.0, stat=Deferred: Connection refused by [127.0.0.1] Oct 30 04:00:04 junkroomserver sm-msp-queue[7445]: m9TF5gpU006108: to=vyoubaby@gmail.com, ctladdr=www-data (33/33), delay=18:54:22, xdelay=00:00:00, mailer=relay, pri=6244126, relay=[127.0.0.1] [127.0.0.1], dsn=4.0.0, stat=Deferred: Connection refused by [127.0.0.1] Oct 30 04:00:04 junkroomserver sm-msp-queue[7445]: m9TExU6a005944: to=vyoubaby <vyoubaby@vyoubaby.com>, delay=19:00:34, xdelay=00:00:00, mailer=relay, pri=6330504, relay=[127.0.0.1] [127.0.0.1], dsn=4.0.0, stat=Deferred: Connection refused by [127.0.0.1]

(note: sensitive information has been obfuscated above)

Can anyone please help me? I’ve been trying for days and weeks to get this running right and still clueless. It seems like sendmail can’t connect to 127.0.0.1 ever since I installed the Webmin relay domains module, procmail, and spamassassin, but like I say, until I did this, I could not receive mail. Now I can receive mail to any domain or user but I just can’t send it at all.<br><br>Post edited by: VyouFinder, at: 2008/10/30 03:37

2

Well, I’m not much of a Sendmail guy (and Postfix is the Virtualmin default) – but just to verify, is Sendmail listening on 127.0.0.1 still?

What shows up when you run:

netstat -an | grep :25

Also, how are you going about sending email? Are you doing that from a remote desktop client, a web interface, or locally from the command line?

If you hadn’t tried it from the command line, I’d be curious if that worked. Something like:

cat /etc/fstab | mail -s "Test Message" user@domain.com

Thanks,
-Eric

3

When I run:

netstat -an | grep :25

I get:

tcp 0 0 0.0.0.0:25 0.0.0.0:* listen

As far as sending mail goes, I am using Webmin’s interface, going into the user’s mailbox and hitting the compose button. I’ve also tried sending mail from a php script that I know works and has worked before when I was unable to receive. No I can receive fine, I just can’t send anymore.

When I tried to send from a command line, I didn’t get the mail to the user’s mailbox… here’s the mail log entry I got from it:

Oct 30 07:31:51 junkroomserver sendmail[5431]: m9UDVpvE005431: from=vyoubaby, size=689, class=0, nrcpts=1, msgid=<200810301331.m9UDVpvE005431@junkroomserver.hsd33.az.comcast.net>, relay=vyoubaby@localhost Oct 30 07:31:51 junkroomserver sendmail[5431]: m9UDVpvE005431: to=polybaby@polybaby.com, ctladdr=vyoubaby (1000/1000), delay=00:00:00, xdelay=00:00:00, mailer=relay, pri=30689, relay=[127.0.0.1] [127.0.0.1], dsn=4.0.0, stat=Deferred: Connection refused by [127.0.0.1]
4

Thanks Joe for your help both here and in your excellent book. It’s nice that people actually respond on this forum and I have found it to be an invaluable resource in learning about servers.

Here’s an update of what I’ve got going on…

I’ve discovered that Postfix should work just like sendmail for all my purposes… and it’s the obvious choice for my system since it seems to mostly work out of the box and is the default for both Ubuntu and Webmin.

Now I can both send mail and receive mail using Webmin’s “read user mail” compose button. No problem, it’s fast and reliable like it should be. Mail is delivered to any email address when sending and when receiving it sorts mail to the appropriate user as well. The problem now is that I can’t get php scripts to use Postfix’s sendmail command to deliver mail. I added the line,

[code:1]
sendmail_path = /usr/sbin/sendmail
[/code:1]
in apache’s php.ini file and that part seems to be working. Problem is, whenever I send scripts through a script it bounces back as blacklisted by RBL as “junk mail.” The link in the bounce email takes me to a page telling me:

my.ip.xx.xx is not listed in the SBL

my.ip.xx.xx is listed in the PBL, in the following records:

* PBL191977

my.ip.xx.xx is not listed in the XBL

On that page there is a link telling me to contact my isp and get it removed, which I did, and which they sound like they’re willing to do… But I don’t want to do it that way if possible. Since I know the system itself can send/receive email, I want it to be able to do this without contacting my isp if at all possible. What if I move the server? What if they decide to blacklist my ip again or if my (dynamic) ip changes… will I have to do this again? I read somewhere that many isp’s will block email sent from php scripts, but how can I get around this so as not to have to alert my isp that I have several servers running here? Is it possible? Again, it seems postfix will send mail natively but not via the sendmail command in a php script in a domain. Why do only my php scripts give me a PBL bounce but regular postfix works great?

After going through all configuration settings I could think of on my new install of Virtualmin, I noticed the setting in:

Webmin>Virtulamin Virtual Servers>Module Configuration>Server Settings

Also update outgoing addresses for mailboxes? and changed this to “yes” based on the help file; what it says about, “These are used to map Unix usernames to email addresses, and can be handy if users send email from scripts.”… so I am thinking this is all that’s missing in my setup, or what might not be working for me. Anyway, when I enabled this feature, and re-checked Virtualmin’s configuration and I got:

The status of your system is being checked to ensure that all enabled features are available, that the mail server is properly configured, and that quotas are active ..

BIND DNS server is installed, and the system is configured to use it.

No sender canonical map file for outgoing addresses was found in your Postfix configuration. You should either add the appropriate map to Postfix, or disable this feature in Virtualmin’s module config page.

It seems that canonical mapping is required for “yes” to be enabled on “Also update outgoing addresses for mailboxes?” but I don’t know how to reliably set up the canonical mapping feature.

After researching on the internet, I edited:

/etc/postfix/main.cf

and added the lines:

[code:1]
sender_canonical_maps = hash:/etc/postfix/sender_canonical
canonical_maps = hash:/etc/postfix/canonical[/code:1]

and was able to re-check and refresh the Virtualmin configuration. So now the canonical maps thing I think is correct.

What about transport mapping? Can this help me?

I am not sure if I have done the above correctly. I’m still not able to end mail from scripts in domains but it feels like I am a lot closer now, thanks to people on this forum.

5

More information - I am sorry to keep posting more but I thought I should post the full message I get from the bounce. Comcast, my isp, tells me that I am not on their RBL. I did their RBL request form and it responded saying they can’t remove my ip from the inbound rbl - keyword inbound, because it’s not blocked, or they say:

Ref: PBL191977

98.192.0.0/11 is listed on the Policy Block List (PBL)

Outbound Email Policy of Comcast for this IP range:

Email sent by Comcast subscribers using a mail program such as Outlook Express are required to send the email through Comcast. To insure your mail program is properly configured, please visit http://www.comcast.net/help/faq/index.jsp?faq=Email117481. If you are a Comcast Commercial Services customer and need support, please contact support_biz@cable.comcast.com

Removal Procedure

Removal of IP addresses within this range from the PBL is not allowed by the netblock owner’s policy.

About The PBL

The Spamhaus Policy Block List (“PBL”) is an international anti-spam system maintained by The Spamhaus Project in conjunction with Internet Service Providers and is used by Internet networks to enforce inbound email policies. The PBL database lists end-user IP address ranges which should not be delivering unauthenticated email to any mail server except those provided for specifically for that customer’s use. The PBL lists only IP addresses (not domains or email addresses).

Keyword in above is unauthenticated. Could it be that I am able to send mail tdirectly through Webmin’s read user mail interface because it’s authenticated, but when using a php script it is not?

I am still hellbent on solving this and appreciate anyone who read through all of the above to help understand.

6

So, what happens if you log into your box over SSH, and you try to send an email from the command line?

Perhaps something like:

cat /etc/fstab | mail -s "Test Email" user@domain.tld

Where "user@domain.tld" is an address that bounced the messages back as being blacklisted.

If that bounces as well, you may need to make sure Postfix is authenticating against your Comcast server whenever it delivers mail.
-Eric

7

I logged onto the local machine, not through ssh but through a keyboard on the machine (is that ok too?) and when I ran:

sudo cat /etc/fstab | mail -s "Test Email" mytestemail@mytestdomain.tld

I got:

The program ‘mail’ can be found in the follwoing packages:

  • mailx
  • mailutils
    Try sudo apt-get install <selected package>
    -bash: mail: command not found

Should I install one or both of these?

8

ok, installed mailutils which I found out includes mailx…

When I send mail to the same myuser@mydomain.tld using

cat /etc/fstab | mail -s "TestMessage" myuser@mydomain.tld

I get the same message… but now I realize this is my mydomain.tld which has been generating this message.

I can send mail to gmail using this method but yahoo and many mail servers will refuse to talk to my mail server, giving the saem RBL message from spamhaus.org as above.

I can still send mail if I use WEbmin’s “read user mail” compose button, but no scripts seem to be able to do it and not with fsstab from the command line.

what does that mean?

9

Update- I figured out something; I was testing to gmail from Webmin’s read user mail module and to other domains using my php script… Now I realize that I cannot send mail to many domains because of being blocked, while others I can send mail to without a problem.

Is this an authentication issue? Suggestions on what to do now?

10

what I would try is to let the PHP script you are using not send through ‘sendmail’ but through ‘smtp server’ thus needing a username and password.

Many php scripts (like joomla) use the simple phpmailer or sendmail feature by default.

11

Ronald, Thanks but I need to be able to use Postfix’s sendmail… Most of my scripts use it exclusively and I don’t want to rebuild all the sites around this problem. It’s also not working when I try to use any form of mailer, ie; sendmail or smtp… I have discovered my results are consistent and in fact I am in an RBL, a blocklist whether using smtp or sendmail, it does not matter.

I’ve seen some posts about using godaddy’s mail or gmail’s mail server by forwarding or routing mail to another mail server. Is this a good option for me? What about doing this for several domains? Getting Comcast to remove the blocklist entry does not look promising so far.

12
I can send mail to gmail using this method but yahoo and many mail servers will refuse to talk to my mail server, giving the saem RBL message from spamhaus.org as above.

And:

Email sent by Comcast subscribers using a mail program such as Outlook Express are required to send the email through Comcast. To insure your mail program is properly configured, please visit http://www.comcast.net/help/faq/index.jsp?faq=Email117481. If you are a Comcast Commercial Services customer and need support, please contact support_biz@cable.comcast.com

So, that’s all the information you need to know. You can’t send mail reliably from your dynamic IP (this has been discussed in numerous threads…you cannot directly send mail reliably from a dynamic IP, for this and several other reasons). You need to send through your ISPs relay host, or use some other relay host. This isn’t optional. Stop banging your head on it, and configure your MTA to send through your ISPs mail server, and be done with it.

13
I've seen some posts about using godaddy's mail or gmail's mail server by forwarding or routing mail to another mail server. Is this a good option for me? What about doing this for several domains? Getting Comcast to remove the blocklist entry does not look promising so far.

Why not use your ISP? That’s what the error message suggested you do. Seems like a good choice to me. But, yes, any other relay host is going to work. You just can’t reliably send mail from a dynamic IP address, so you need for your mail server to always relay outgoing mail to another host that is a static IP and not in any RBLs.

Getting Comcast to remove the blocklist entry does not look promising so far.

It’s not Comcast that has blacklisted the IP range. You are on a dynamic IP block–one that is used by Comcast users. Because the IP range is a public dynamic IP range, no one can ever reliably determine who you are, SPF records can never be reliable, and so it is on several RBLs. You cannot reliably send email directly from a dynamic IP address. So, as the error message suggested, you must relay through a server on a static IP address. In Sendmail this is the “smart host” configuration, and in Postfix it is called “relay host”.

Also, I noticed when reading through this thread again that you’d installed the mail relay module. I strongly suspect you do not need/want this module, and trying to use it will only break your configuration. It has a very specific purpose, and I’m not getting the impression that it would have any useful purpose in your deployment.

14

I’m pretty confident the problem is in fact an RBL blacklist problem at this point. I have done just a little research on using Godaddy’s SMTP server, but theirs is limited and costs per relay. From what I’ve read so far, Gmail offers free SMTP relaying, so I am starting down that path here:

http://souptonuts.sourceforge.net/postfix_tutorial.html

It might not be the solution I was originally looking for, but I’ve learned a lot along the way and by the time I get the ability to send outgoing email, I’ll have learned even more. Thanks for the responses in getting to the root of the problem which as it turns out was my testing procedure.

15

You’re definitely right about the dynamic ip being within a range of blocked ip’s. It all makes sense now. Finally. I should’ve noticed it only accepting a static ip, when setting up my dynamic DNS script and dns records.

Also, I noticed when reading through this thread again that you'd installed the mail relay module.
  • On a previous installation I installed a special (maybe even third party) Webmin sendmail relaying module… Since then I have done a clean install with Postfix. So the Relaying module is gone. In Postfix, I added:

[code:1]
/etc/postfix/main.cf

added two lines:

sender_canonical_maps = hash:/etc/postfix/sender_canonical
canonical_maps = hash:/etc/postfix/canonical
[/code:1]

but didn’t set anything up further, so I am thinking it’s not affecting things. Are you saying this will cause problems for me?

Why not use your ISP? That's what the error message suggested you do. Seems like a good choice to me. But, yes, any other relay host is going to work
  • I never thought of that. I guess I probably have do access to mail there. I sort of prefer going with gmail, in case for example I move the server to another isp or location or something, then the gmail smtp forward is already in place. The other reason to consider gmail might be that it provides the possibility of the gmail web email tool for some domain users on my server - if they should want it. I don’t know that using my isp’s smtp server to relay has any benefits and when I looked through their help stuff they had a section on buying per relay stuff. I hardly use any email at all but I just don’t like the idea of their limiting me and keeping me under their thumb in any way.

Maybe I will try them both. I am just relieved to have figured out the root of the problem and to learn that there’s a solution that will work for me.

16

It’s not an option… Most of my scripts use the sendmail command exclusively.

However, I’ve learned that it’s due to my being on a dynamic ip.

Problem resolved and will be relaying to gmail or other smtp mail server instead.