Server has suddenly stopped sending or receiving email

Virtualmin
1

Hi all,

Without being touched for weeks, my Virtualmin server has suddenly stopped receiving email, and upon sending returns the error “The sender address person@domain.com was rejected by the server domain.com”

Email sent to the server is not being returned to the sender, so I am unclear as to exactly when the problem occurred, as it’s a server that does not receive very much email so a few days passed before I noticed that nothing was arriving.

I do notice that in var/log/ the maillog files are absolutely huge:
maillog 288.9MB
maillog.1 531.2MB
maillog.2 530.2MB
maillog.3 528.6MB
maillog.4 528.8MB

so obviously large amounts of errors are being written in there. I am downloading one as I type, it will take all night and then I face the challenge of finding a text editor that can open a file that large.

I was just hoping someone on here may be able to point me in the right direction to start to troubleshoot the problem.

Cheers,

loyalwhite

2

Well, you may want to start by restarting Postfix, and then look in the mail logs to see if it generated any errors during the restart.

Next, you can try sending an email to your server, and review the logs on your server during that time… what errors do you see?

Lastly, you may want to go into Virtualmin, and click System Settings -> Re-Check Config, and see if it notices anything out of the ordinary.

-Eric

3

OK, I’m getting myself into a mess here.

I finally downloaded the 230MB maillog file, and it contained an entry like this literally every second (which I guess explains the size of the files:

Apr 28 00:22:48 server88-208-236-226 postfix/pickup[14929]: warning: 2605BE2901: message has been queued for 68 days Apr 28 00:22:48 server88-208-236-226 postfix/pickup[14929]: 2605BE2901: uid=0 from=<root> Apr 28 00:22:48 server88-208-236-226 postfix/cleanup[22677]: 2605BE2901: message-id=<20110427232248.2605BE2901@server88-208-236-226.live-servers.net> Apr 28 00:22:48 server88-208-236-226 postfix/cleanup[22677]: warning: 2605BE2901: write queue file: No space left on device

I googled the “write queue file: No space left on device” error and someone suggested running df -h

I did so and got this:


Filesystem            Size  Used Avail Use% Mounted on
/dev/sda2              16G  2.0G   13G  14% /
/dev/sda5             4.8G  4.5G   56M  99% /var
/dev/sda3             204G   16G  178G   8% /home
/dev/sda1              76M   30M   43M  41% /boot
tmpfs                 1.9G     0  1.9G   0% /dev/shm

Seeing that the /var filesystem was at 99% capacity, in a moment of madness I deleted the maillog files, all five of them, and made a new maillog file at the same location. Nothing is now being written into that file, so Eric, I followed your suggestion and restarted Postfix, but now I have an empty maillog and I don’t know what to do to get Postfix to write into it again.

I’m stuck. Any help would be sincerely appreciated.

4

Howdy,

Postfix isn’t actually what writes to the logs, that would be syslog.

So you may want to try restarting the syslog daemon, it may have gotten confused when the mail log dissapeared.

-Eric

5

OK, I rebooted the box and as you suggested Eric, this caused the log to be written successfully again. Now I am receiving email messages (all the ones sent but not previously received have now suddenly come through) and I can send again with no errors.

I do wonder what the hell caused those maillog files to become so bloated, which seems to fill that partition and cause mail to fail. I guess I’ll just have to keep an eye out for it happening again.

Thanks for your help, Eric.