Not able to make sense of the big list, did it without verbose and got this.
missing c /etc/httpd/conf.d/welcome.conf
S.5....T c /etc/httpd/conf/httpd.confHowdy,
Well, you’ll need to peek in the rpm manpage to get a list of what all that means 
In short, though, it’s saying the only files different now from when the package was installed are those two config files.
It’s saying the welcome.conf file isn’t there (which is fine), and that the httpd.conf file has changed (which we know).
So long story short – outside of those two config files, everything else relating to the httpd package is completely normal.
If you’re concerned, you could expand your search to other related packages, such as PHP… you’d want to be on the lookout for anything it says about binary files, I wouldn’t be concerned with config files.
-Eric
php gives only this:
[root@server ~]# rpm -V php-cli
prelink: /usr/bin/php: at least one of file's dependencies has changed since prelinking
S.?..... /usr/bin/php
prelink: /usr/bin/php-cgi: at least one of file's dependencies has changed since prelinking
S.?..... /usr/bin/php-cgiIn other news, I have a funny feeling it might have been the maxclients was set way too high.
Tuning this back I have only seen a maximum of 6 child processes running (it can spawn up to 19 (well 12-15 depending on if it uses the min or max spare clients settings).
I have even seen the mem use drop after getting to around 20% back to around 17% and Apache seems to be behaving.
This is the script I borrowed to tell me the recommended MaxClients
#!/bin/bash
echo "This is intended as a guideline only!"
if [ -e /etc/debian_version ]; then
APACHE="apache2"
elif [ -e /etc/redhat-release ]; then
APACHE="httpd"
fi
RSS=`ps -aylC $APACHE |grep "$APACHE" |awk '{print $8'} |sort -n |tail -n 1`
RSS=`expr $RSS / 1024`
echo "Stopping $APACHE to calculate free memory"
/etc/init.d/$APACHE stop &> /dev/null
MEM=`free -m |head -n 2 |tail -n 1 |awk '{free=($4); print free}'`
echo "Starting $APACHE again"
/etc/init.d/$APACHE start &> /dev/null
echo "MaxClients should be around" `expr $MEM / $RSS`php gives only this:
Those may actually okay too, I believe both of those are just symlinks…
Tuning this back I have only seen a maximum of 6 child processes running
Well, while I still find all this a little odd, it’s great that it’s working for you now
Usually, Apache wouldn’t just silently crash during a low-memory situation; and while it’s possible that the kernel’s OOM-Killer could have killed Apache to prevent the server from dieing, that would have showed up in both the system logs as well as the dmesg output.
I don’t recall if you’re using Virtualmin Pro or GPL – but if you’re using Pro, you may want to take a peek at the system statistics during the time Apache has been crashing to see if your RAM was all used up.
Either way though, I’m glad you haven’t seen this issue come up lately!
-Eric
Best thing too, while I’ve been playing with all of this at work, 3 work mates have gone “Oh, are you doing hosting? I need some webspace…”
So if this goes nicely for the next week… YIPPIE!!
I’ve just restored a forums with about 50-100 members on a given week posting and a core group of 10-12 people on daily. With over 55K posts… so I’ll see how it behaves now.
Nup - didn’t work.
Apache still crashing / restarting on me and no errors jump out and say why. 
At least the PID isn’t changing (monit tells me when it changes).
Ok - going out of my mind here.
I restarted Apache as I had 1 website that wasn’t responding to http requests (even after restarting) but it came good on it’s own…
Then within 30 minutes, Apache had restarted again, on it’s own… and Monit advertised the PID had changed.
NO errors as to why, Monit didn’t do it as it alerts me if it restarts it…
I’m tempted to scratch what I have and re-install and try again…
Well, to try another avenue… you say this typically happens right after a graceful restart?
Apache should be able to come back online after such a restart. If you manually try a graceful restart, does that work? Or does Apache fail?
Also, as Scott mentioned earlier, you may want to tinker with increasing the Apache loglevel. Setting it to “crit” as you did would actually display less logs, showing just “critical” errors rather than all messages. You may want to experiment with “info” and “debug”.
If you’re discovering that Apache fails each time you do a graceful restart, that’s no good That could certainly be the issue then, which makes the key to figure out what’s causing Apache to not start back up…
If you can reproduce the issue by performing graceful re-starts, perhaps increasing the log messages would explain why it’s failing.
-Eric
Yeah - I’ve set it to Debug and I’ll let it go and see what happens.
Thing I noticed is the /var/log/httpd/error_log doesn’t get any errors in it, but the virtualmin error log seems to capture it all.
So I might download those and start trawling through it for what’s going on (didn’t realise the errors were being redirected to it).
Trawled and nothing…
Gotta wait for a proper crash I guess while debug is on. Just a bunch of errors from Wordpress due to duplicate IP entries in the database (spam bot registrations), there were some error from an old simple gallery script I had up. But I’ve since removed that just incase (some undeclared functions and what not).
I DID get the script about 2-3 years ago, nothing fancy. And no updates to it since I got it.
Time will only tell now.
So far so good.
I have a funny feeling it was this script…
fingers crossed
If I get more than 7days uptime then I KNOW it’s resolved and that the problem was a fairly outdated script (and the author’s e-mail no longer active).
Pity, was a very simple gallery too.