Webalizer putting a busy server on its knees. Couldn't it be made nicer ?


An image to start the discussion: http://imgur.com/9zeM2jk

Here’s the thing, I noticed that my server (a nice dedi, with good disks - 2 Raid-1 SSDs for the system and two Raid-1 SATAS for home -, around 8 Gb unused RAM, and a good processor, an AMD Opteron™ Processor 4334 , 6 cores, almost never going over one third of its capacity in usual usage) is thrown on its knees whenever webalizer feels like it.

With 6 cores, I find myself having a load average in the 40-50 instead of the usual 2, 3 or 4.

And, in those occasions, who do I see peaking and showing the tip of its nose ?

(Not always the same PID, but always the same thing)

At first I suspected this was a weird random level 7 attack, as the number of active CPU threads grew from around 400 to 600-650, but seems more like these are visitors not getting in time what they’re expecting because the server is too busy processing its log files and generating stats.

(Well, I could be mistaken of course, but it bloody looks like this, outside of those crazy peaks webalizer is never sitting on a throne on top of the CPU usage charts)

Candid questions, why not give all processes with webalizer a superior nice level, for instance ?

Or make the program first do a copy of the log files and then process those copies instead of the originals. Because, well, when there are constant updates on a log file, it may not be so surprising if the process has trouble reading it while the system is busy updating it in the same time…

Well, that’s it for the bug report, and also for the improvement suggestion, to make it easier on the system…

Good day everyone!