Procmail and spamassassin woes.

Virtualmin
21

Hi Sean,

After much debugging on your system, I discovered that your procmail behaves differently to the one everywhere else - it runs /etc/procmailrc as the user receiving mail right from the start, rather than only when DROPPRIVS=yes is encountered.

When I copied in the exact same version of procmail from my Ubuntu system (3.22) to /usr/bin/procmail, it started working just fine! My guess is that the version built by Gentoo has some strange patches that cause this. Bizarre …

Nick - are you on Gentoo as well?

22

Jamie

Nick - are you on Gentoo as well?

Fraid not - mine is a centos 4.6 box which was upgraded prob from centos 4.1 as its original build.

I can give you access if you need it? or perhaps you can tell me what I am looking for?

Nick

23

One thing to check would be the permissions on /usr/bin/procmail-wrapper - they should be -rwsr-sr-x as shown by ls -l.

Also, make sure that /etc/postfix/main.cf has a line like :

mailbox_command = /usr/bin/procmail-wrapper -o -a $DOMAIN -d $LOGNAME

24

Jamie

I can confirm both are correct:

-rwsr-sr-x 1 root root 6851 Aug 3 2006 /usr/bin/procmail-wrapper

main.cf: mailbox_command = /usr/bin/procmail-wrapper -o -a $DOMAIN -d $LOGNAME

Nick

25

I’d be happy to login to your system too to take a closer look - if that’s OK, email me your login details at jcameron@virtualmin.com

26
When I copied in the exact same version of procmail from my Ubuntu system (3.22) to /usr/bin/procmail, it started working just fine! My guess is that the version built by Gentoo has some strange patches that cause this. Bizarre ..

So it’s the exact same version, but just seems to run differently? That is bizzare. I can look further into the Gentoo ebuild and see why it’s like that. Do you have any idea what we can possibly do to check about whether procmail runs like this? Maybe you can add that check to your install script.

Did you change anything else on the system besides the binary?

Thanks so much for your help, this has been a head scratcher for Months! Once again, I am much in debt to you! :slight_smile:

The last only weird problem I have now is that I updated syslog-ng and logrotate a couple weeks ago. This seems to have caused Postfix to once in awhile, stop writing to the mail.log file. I think this usually happens on the logrotate shedule. Should I add a /etc/init.d/postfix reload in the after actions for that rotation?

27

My mail headers are finally showing the results:

X-Spam-Flag: No X-Spam-Score: -2.3 X-Spam-DCC: _DCCB_: _DCCR_ X-Spam-Checker-Version: SpamAssassin 3.2.1-gr1 (2007-05-02) on salvonexus.spindlex.com X-Spam-Level: X-Spam-Status: No, score=-2.3 required=4.0 tests=AWL,BAYES_00 autolearn=ham version=3.2.1-gr1 X-Spam-Pyzor:

Awesome! Thanks again Jamie!

28

Yes, it was the same Procmail version number.

Oh, I also noticed that some Webmin programs were using /var/webmin as their log directory (which doesn’t exist), while others used /var/log/webmin . This is why the lookup-domain-daemon.log file wasn’t being updated. Once I linked /var/webmin to /var/log/webmin , all was well.

29

After looking at Nick’s system, I found the real cause of this problem, and it turned out to be a bug in procmail !

It seems that if the default mail directory /var/mail doesn’t exist, procmail’s code will stupidly switch to the user who is receiving email right away, causing most of /etc/procmailrc to fail. When I created the /var/mail directory on his system, it started working again.

I’m going to file a bug with the procmail developers about this …

30

Jamie

Thanks for all the work on this - much appreciated - I thought it was all something I had done. Emails are now properly being tagged!

Nick

31

Jamie,

Great detective work! I see that the procmail bug couldn’t have affected my system since I had the /var/mail dir. Did you find this through the source or just investigating? Maybe we can make a patch.

Anyways, thanks again for your help.

32

I’m betting Jamie used his favorite system-level sleuthing tool, strace…but that’s just a guess.

33

Yeah, strace knows all … and it was able to point me in the right direction in this case. I also had to look at the procmail source code though.

In Sean’s case, I’ve realized that procmail was looking for mail in the ~/.maildir directory, and incorrect switching IDs when that was not found. This must have been a path compiled into the Gentoo version of procmail, which is why it started working when I replace procmail with the copy from my system (which looks for /var/mail , which exists on Sean’s system).

34

Ok, someone please answer me if they have an idea. This is driving me crazy.

Here is what I think is really happening.

This is my /etc/procmailrc

[code:1]

Use maildir-style mailbox in user’s home directory

VERBOSE=yes
LOGFILE=/var/log/procmail.log
TRAP=//etc/webmin/virtual-server/procmail-logger.pl
:0wi
VIRTUALMIN=|/etc/webmin/virtual-server/lookup-domain.pl $LOGNAME
:0

  • ?/usr/bin/test "$VIRTUALMIN" != ""
    {
    INCLUDERC=//etc/webmin/virtual-server/procmail/$VIRTUALMIN
    }
    ORGMAIL=$HOME/Maildir/
    DEFAULT=$HOME/Maildir/
    DROPPRIVS=yes

:0

  • ^X-Spam-Status: Yes
    $DEFAULT
    [/code:1]

And looking at the procmail log, I see this.

[code:1]
procmail: Assigning "TRAP=//etc/webmin/virtual-server/procmail-logger.pl"
procmail: Assigning "VIRTUALMIN="
procmail: Executing "/etc/webmin/virtual-server/lookup-domain.pl,joesomeone.mydomain"
procmail: Executing "/usr/bin/test,!=,"
procmail: Non-zero exitcode (1) from "/usr/bin/test"
procmail: No match on "/usr/bin/test != "
procmail: Assigning "ORGMAIL=/home/windish/homes/amy/Maildir/"
procmail: Assigning "DEFAULT=/home/windish/homes/amy/Maildir/"
procmail: Assigning "DROPPRIVS=yes"
procmail: Assuming identity of the recipient, VERBOSE=off
procmail: No match on "^X-Spam-Status: Yes"
procmail: Assigning "PATH=/home/mydomain/homes/joesomeone/bin:/bin:/usr/bin:/usr/local/bin"
procmail: Assigning "LASTFOLDER=/home/mydomain/homes/joesomeone/Maildir/new/1201644084.13825_0.myserver.mydomain.com"
procmail: Notified comsat: "amy.windishagency@0:/home/mydomain/homes/joesomeone/Maildir/new/1201644084.13825_0.myserver.mydomain.com"
From somebloke@outheresomehwere.com Tue Jan 29 16:01:24 2008
Subject: This is the story of my life
Folder: /home/mydomain/homes/joesomeone/Maildir/new/1201644084.13825_0.myserver.mydomain.com 9016
procmail: Assigning "EXITCODE=0"
procmail: Executing "//etc/webmin/virtual-server/procmail-logger.pl"
Time:1201644084 From:«»somebloke@outheresomehwere.com To:joesomeone@mydomain.com User:joesomeone.mydomain Size:9062 Dest:/home/mydomain/homes/joesomeone/Maildir/new/1201644084.13825_0.myserver.mydomain.com Mode:None

[/code:1]

Now look at this…

procmail: Assigning "VIRTUALMIN=" procmail: Executing "/etc/webmin/virtual-server/lookup-domain.pl,joesomeone.mydomain"
VIRTUALMIN is not getting assigned. And the lookup-domain.pl script is being fired AFTER the assignment. This makes no sense to me. As far as I understand Procmail, =| will run the script following it, wait for it to return with data, and then set it to the variable to the left. Is this not happening for some reason? Is there some magic Procmail etting that is set somewhere that I don't know about?

Any ideas?

Anyone?

Anyone?

35

Here is my current procmailrc:

[code:1]

Use maildir-style mailbox in user’s home directory

VERBOSE=yes
LOGFILE=/var/log/procmail.log
TRAP=//etc/webmin/virtual-server/procmail-logger.pl
:0wi
VIRTUALMIN=|/etc/webmin/virtual-server/lookup-domain.pl $LOGNAME
:0

  • ?/usr/bin/test "$VIRTUALMIN" != ""
    {
    INCLUDERC=//etc/webmin/virtual-server/procmail/$VIRTUALMIN
    }
    ORGMAIL=$HOME/Maildir/
    DEFAULT=$HOME/Maildir/
    DROPPRIVS=yes

:0

  • ^X-Spam-Status: Yes
    $DEFAULT
    :0
    $DEFAULT

[/code:1]

36

Here is my current procmailrc:

[code:1]

Use maildir-style mailbox in user’s home directory

VERBOSE=yes
LOGFILE=/var/log/procmail.log
TRAP=//etc/webmin/virtual-server/procmail-logger.pl
:0wi
VIRTUALMIN=|/etc/webmin/virtual-server/lookup-domain.pl $LOGNAME
:0

  • ?/usr/bin/test "$VIRTUALMIN" != ""
    {
    INCLUDERC=//etc/webmin/virtual-server/procmail/$VIRTUALMIN
    }
    ORGMAIL=$HOME/Maildir/
    DEFAULT=$HOME/Maildir/
    DROPPRIVS=yes

:0

  • ^X-Spam-Status: Yes
    $DEFAULT
    :0
    $DEFAULT

[/code:1]

37

Hey Jamie,

When you run lookup-domain.pl as root from the command line, does an entry get logged to that file?

No, it does not appear that anything gets logged when I run lookup-domain.pl as root. That is if you mean in the /var/log/webmin/lookup-domain-daemon.log.

My mailbox command for postfix has always been:

[code:1]mailbox_command = /usr/bin/procmail-wrapper -o -a $DOMAIN -d $LOGNAME[/code:1]
This is how I configured it from installation.

And this is how my procmail-wrapper looks:

[code:1]-rwsr-sr-x 1 root root 8266 Oct 22 00:37 /usr/bin/procmail-wrapper[/code:1]

I did have to make my procmail-wrapper on my own. This Virtualmin installation for the most part was performed manually. Here is the source I used for the wrapper (found on this forum):

[code:1]

procmail-wrapper.c

#include "stdio.h"
int main(int argc, char **argv)
{
setuid(geteuid());
setgid(getegid());
execv("/usr/bin/procmail", argv);
}
[/code:1]

Do I need to have postfix run as another users? Currently it runs as postfix.

Here is a list of my usual postfix processes that are running.

[code:1]salvonexus seanwolfe # ps aux | grep postfix
postfix 14666 0.0 0.0 41036 2532 ? S Jan31 0:02 qmgr -l -t fifo -u
postfix 14678 0.0 0.0 40700 2224 ? S Jan31 0:01 anvil -l -t unix -u
postfix 14681 0.0 0.0 40704 2268 ? S Jan31 0:00 tlsmgr -l -t unix -u
postfix 16548 0.0 0.0 51912 4128 ? S 16:20 0:00 smtpd -n smtp -t inet -u -o smtpd_sasl_auth_enable yes
postfix 16554 0.0 0.0 51932 4176 ? S 16:20 0:00 smtpd -n smtp -t inet -u -o smtpd_sasl_auth_enable yes
postfix 16561 0.0 0.0 40952 2648 ? S 16:21 0:00 cleanup -z -t unix -u
postfix 16850 0.0 0.0 40708 2148 ? S 16:29 0:00 pickup -l -t fifo -u
postfix 17037 0.0 0.0 40864 2524 ? S 16:32 0:00 smtp -t unix -u
postfix 17171 0.0 0.0 40964 2740 ? S 16:35 0:00 local -t unix
[/code:1]

38

Okay, I think we are getting somewhere. I don’t seem to have an init script for lookup-domain-daemon.pl.

I do see /etc/webmin/init/lookup-domain.sh though.

[code:1]salvonexus seanwolfe # cat /etc/webmin/init/lookup-domain.sh
#!/bin/sh

Daemon for quickly looking up Virtualmin servers from procmail

case “$1” in
‘start’)
/usr/local/webmin/virtual-server/lookup-domain-daemon.pl
RETVAL=$?
;;
‘stop’)
kill cat //var/log/webmin/lookup-domain-daemon.pid
RETVAL=$?
;;
‘restart’)
$0 stop ; $0 start
RETVAL=$?
;;
*)
echo “Usage: $0 { start | stop }”
RETVAL=1
;;
esac
exit $RETVAL[/code:1]

But, I do see the process running.

[code:1]salvonexus seanwolfe # ps aux | grep lookup-domain-daemon
root 13730 0.0 1.1 100264 69412 ? Ss Jan28 0:16 /usr/local/webmin/virtual-server/lookup-domain-daemon.pl
[/code:1]

Maybe the script isn’t running with the correct params?

But then looking to see if it is listening to a port, i do a netstat:

[code:1]salvonexus seanwolfe # netstat -tap | grep 13730
tcp 0 0 localhost:11000 : LISTEN 13730/lookup-domain [/code:1]

To me, everything is where it should be. If you think I should, let you in to debug it, you can email me your public key file, and I’ll create an account that can su in.

39

Okay, I think we are getting somewhere. I don’t seem to have an init script for lookup-domain-daemon.pl.

I do see /etc/webmin/init/lookup-domain.sh though.

[code:1]salvonexus seanwolfe # cat /etc/webmin/init/lookup-domain.sh
#!/bin/sh

Daemon for quickly looking up Virtualmin servers from procmail

case “$1” in
‘start’)
/usr/local/webmin/virtual-server/lookup-domain-daemon.pl
RETVAL=$?
;;
‘stop’)
kill cat //var/log/webmin/lookup-domain-daemon.pid
RETVAL=$?
;;
‘restart’)
$0 stop ; $0 start
RETVAL=$?
;;
*)
echo “Usage: $0 { start | stop }”
RETVAL=1
;;
esac
exit $RETVAL[/code:1]

But, I do see the process running.

[code:1]salvonexus seanwolfe # ps aux | grep lookup-domain-daemon
root 13730 0.0 1.1 100264 69412 ? Ss Jan28 0:16 /usr/local/webmin/virtual-server/lookup-domain-daemon.pl
[/code:1]

Maybe the script isn’t running with the correct params?

But then looking to see if it is listening to a port, i do a netstat:

[code:1]salvonexus seanwolfe # netstat -tap | grep 13730
tcp 0 0 localhost:11000 : LISTEN 13730/lookup-domain [/code:1]

To me, everything is where it should be. If you think I should, let you in to debug it, you can email me your public key file, and I’ll create an account that can su in.

40

Hi Sean

Procmailrc is:

[code:1]# Use maildir-style mailbox in user’s home directory
VERBOSE=yes
LOGFILE=/var/log/procmail.log
TRAP=//etc/webmin/virtual-server/procmail-logger.pl
:0wi
VIRTUALMIN=|/etc/webmin/virtual-server/lookup-domain.pl $LOGNAME
:0

  • ?/usr/bin/test "$VIRTUALMIN" != ""
    {
    INCLUDERC=//etc/webmin/virtual-server/procmail/$VIRTUALMIN
    }
    ORGMAIL=$HOME/Maildir/
    DEFAULT=$HOME/Maildir/
    DROPPRIVS=yes

:0

  • ^X-Spam-Status: Yes
    $DEFAULT
    :0
    $DEFAULT
    [/code:1]
Is your virtualmin config installed in /etc/webmin/virtual-server directory? Is it elsewhere?

Yep in that directory!

What does /etc/webmin/virtual-server/lookup-domain.pl look like?
[code:1] #!/usr/bin/perl open(CONF, "/etc/webmin/miniserv.conf"«»); while(<CONF>«») { $root = $1 if (/^root=(.*)/); } close(CONF); $ENV{'WEBMIN_CONFIG'} = "/etc/webmin"; $ENV{'WEBMIN_VAR'} = "/var/webmin"; chdir("$root/virtual-server"«»); exec("$root/virtual-server/lookup-domain.pl", @ARGV) || die "Failed to run $root/virtual-server/lookup-domain.pl : $!"; ~ [/code:1]
What is root= set to in /etc/webmin/miniserv.conf?
[code:1]root=/usr/libexec/webmin [/code:1]

Permissions on file are:

[code:1][code][root@www ~]# ls -la /etc/webmin/virtual-server/lookup-domain.pl
-rwxr-xr-x 1 root root 360 Aug 2 2006 /etc/webmin/virtual-server/lookup-domain.pl
[/code][/code:1]

Anything else you can suggest? I am kind of at a loss with this one! having traced it as far as lookupdomain failing.

Nick