I’ve been using an account on Rackspace Cloud to host a few clients for which I develop websites. Over the past year and a half of doing so, I seem to have issues where most of the applications on my server crash, requiring a hard reboot. Sometimes it happens twice in a week, other times it happens several months apart. My clients are starting to get angry and I’ve been at a loss as to what could be causing it… until I noticed this in my syslog:
After a little bit of Googling, I found out that miniserv.pl belongs to Webmin, and that while Webmin alone doesn’t require much ram, Virtualmin does when it is used for various common functions. Thinking back, I believe these crashes may occur within a day or so of using Virtualmin.
My cloud slice is running Ubuntu at 512 MB of RAM with a gig of virtual memory. I’m running Apache, PHP, MySQL, Mailman, and have about 10 sites running on Wordpress.
Could accessing Virtualmin be causing the crashes that bring all of my clients’ sites down periodically? Anything in the logs that I can look for to see for sure? I’m more of a developer than a pro sysadmin.
I am doing a similar thing as you, but at Elastichosts, not Rackspace.
When you say “My cloud slice is running at 512 MB of RAM with a gig of virtual memory” - I see problems! You need lots of RAM with cloud servers and you have to forget about swap altogether IMO (which is what the Apache folks say anyway). I have 4Gb RAM for around 150 virtual servers. I’m not yet sure that’s enough (see http://www.virtualmin.com/node/21079)
Are you throttling your services to stop spikes taking you down? i.e. PostFix - limit the maximum connections? Apache - limit MaxClients? With only 512Mb have you tried calculating how many Apache processes you can spawn before your kernel starts to panic?
From my experiences, I have tried and do have a vitrualmin site running on a t1.micro instance on AWS. If your using more that 2/3 of the ram you will see problem with your sites. And if your running over your memory limit getting into virtual memory, you will have issues… About 2 sites seem the be the max with them using databases it what I think you find… I think even if you turn off webmin and usermin, you see the same issues. Restart with lots of ram, and all your issues goes away… except paying the aws bill that jumps from 0 to $8 a day…
Could accessing Virtualmin be causing the crashes that bring all of my clients’ sites down periodically? Anything in the logs that I can look for to see for sure? I’m more of a developer than a pro sysadmin.
It’s rather unlikely that Virtualmin itself is the cause of the issues you’re seeing – you may want to take a peek in your logs in /var/log. As some other folks have already mentioned, you may be running low on RAM, and there’s a number of services running that could be the cause of that.
The message you found above is indeed a problem – that shows that the miniserv.pl process tried to allocate RAM, and wasn’t able to.
However, that doesn’t mean miniserv.pl was the culprit – the key is to determine what pushed your server to have so little RAM to begin with?
One thing you might do is review the Virtualmin on Low-Memory systems guide, which might help you in reducing your system’s memory consumption:
I haven’t really done much in the way of performance optimizations, as I don’t know much about it. Is there a good way to calculate the number of processes Apache can spawn? I’m looking at this article:
…and it says a simple calculation for MaxClients could be (Total Memory - Operating System Memory - MySQL memory) / Size Per Apache process.
It looks like I currently have 10 Apache processes running, with a VSZ of roughly 269000 each. Is there a good way to calculate what performance settings I should use? I could scale up the amount of RAM available to my cloud slice, but I’m not sure how big to make it. Up to 2048 MB of RAM is probably fine, but past that it starts to get pricey.
I’d guess (and I’m not sure I know what I’m talking about!) that if you go for 2048MB RAM, then perhaps set MaxClients & ServerLimit to 30/40 you might be OK. If you’re regularly hitting that limit you’ll see “server reached MaxClients setting, consider raising the MaxClients setting” in your Apache error log. I also set the Postfix connection limit to e.g 20-40 (A wave of spam hitting your server will require a lot of RAM for SpamAssassin and ClamAV. I’ve only just found out how Postgrey is my friend…).