This problem started some months ago. I did read almost every post about it and similar situations in here and other forums. I cannot seem to fix my problem with any solution that applied to other cases.
I am using FCGid, so each php executes as each virtual server owner. The number of running php-cgi keeps getting bigger, until all ram is eaten and other processes start shutting down (usually the first one is clamd)
My server got 9 Gb of assigned RAM. Webmin manages about 88 websites.
After much tinkering, my current httpd.conf is:
Timeout 60
KeepAlive On
MaxKeepAliveRequests 75
KeepAliveTimeout 3
StartServers 32
MinSpareServers 3
MaxSpareServers 32
ServerLimit 256
MaxClients 128
MaxRequestsPerChild 4096
StartServers 32
MaxClients 128
MinSpareThreads 25
MaxSpareThreads 75
ThreadsPerChild 25
MaxRequestsPerChild 4096
And (I did reset apache some 10 minutes ago) my server's memory usage is:
Private + Shared = RAM used Program
In a couple of hours, php-cgi will eat up to 8gb of ram or so, and then “bad things will start to happen” (other processes start shutting down).
I am out of ideas. My knowledge of Linux and server intricacies is sadly enough not as good as it should. I am hoping one of the experts over here can suggest something in order to point me into the right direction.
Note: Memory limit was at 128M … I downsized it into 64M but then several wordpress websites started showwing 500 errors, so I found the 72M was a low working value. Not that it did have any effect on my problem, anyways.
The rest of the parameters are as “they were” (I assume, default values for virtualmin / webmin).
About fcgid … maybe I don’t know where to check that !
Inside httpd.conf there is no mention of mod_fcgid.c …
Also, inside php.ini there is no mention of fcgid either.
Inside /etc/httpd/conf.d/ I found a fcgid.conf which is quite small:
As a side note, right now, 3 hours after my first post, memory and count usage on php-cgi has escalated into:
(private - Shared - Total RAM used - Program) 7.6 GiB + 29.0 MiB = 7.6 GiB php-cgi (130)
On Centos 7 fcgid settings can be found in /etc/httpd/conf.d/httpd.conf or /etc/httpd/conf.d/fcgid.conf. If you are using different distro then probably there will be some differences so try to find appropriate match for your OS. I remember on Centos fcgid was left to the default settings which are bad, with 1-2 settings inside httpd.conf (not even sure about this) and same for fcgid.conf.
Aside of this, you should check Apache access/error log files and (if missing) install network-tools so you can see how many active connections you have. It could even be someone is trying to mess around with your server with DOS, slow get/post requests or in some other way. Either way is worth to check your current active connections and see what is going on.
Could you post here content of your fcgid.conf file and parts of httpd.conf containing fcgid lines (if any). It would help to know what kind of server is that (VPS or dedicated), how many cores and total ram.
It is a dedicated server with 16 gb RAM, an Intel SSD (3700 series) and an INTEL Xeon E3-1240 processor running the latest proxmox virtualization environment.
Inside proxmox I have a qemu based virtual machine which has the linux + virtualmin official ISO installed. Everything is updated to the latest official packages.
I already placed the entire fcgid.conf file content (minus comments) on my first answer to you. It is just 4 lines of settings:
I am suspecting that the relevant configuration lines are missing (if so, such is the way it came inside the official virtualmin iso - centos download I installed originally).
I would think that there are some commands that may cap the amount of php5.fcgi that each virtual server can span. If so, then I would -hopefully- be able to control the amount of ram being used by them.
Just so you know, this isnt “one-win-button” for all and i’m sure you will need to modify this settings, still right now your primary problem is Apache taking all your memory so best would be to go step by step.
Once done be sure to re-check your httpd.conf file for any Fcgid* lines and remove them (backup old file!).
Hi Diabolico! Thanks for the configuration for fcgid.conf . I did paste it and now reset Apache. I am eagerly awaiting for it to work a couple of hours in order to see if it improves. I am also searching for description on each configuration line you gave me, so I can better understand and eventually tweak each one.
Inside httpd.conf I finally found some Fcgid related config lines.
Each declared website includes the following lines:
I did not remove any of those lines yet. I suspect that I may be spawning two fcgi-bin threads when only one is necessary, but then again it probably is my ignorance. As I type this, I am already searching for configuration examples and explanations on the web about how Fcgid needs to be configured / specified inside httpd.conf but of course, any idea, suggestion or comment is very much appreciated.
FcgidMaxRequestLen you already have in fcgid.conf file so you can delete this line.
AddHandler*, AddType* and FCGIWrapper* leave it.
Settings i gave you (fcgid.conf) need to be modified depending on your server spec and traffic on your website(s), so this numbers are not set in stone. After you see the situation in few hours you can start the adjustments.
-Happily enough, I suppose- now memory is on the low side (I am at about 30% of real memory usage).
On the CPU side, It is oscillating between 14% and 30% usage.
I will closely follow server behavior today and tomorrow (tomorrow it will receive heavier traffic).
Thanks for your help, and of course if you notice something odd on the parameters I touched, please tell me so I can be aware of it!
There is no harm if you leave settings with default values as this could change in some future software update, but its up to you.
For the CPU and the rest its just a game of balancing everything. For example, with 16GB memory you could easily dedicate 8-10GB (or more) for Apache and MySQL caching. This alone should decrease the overall pressure on your server but it will use much more memory. Still to have 10+GB of unused memory isnt doing any good and you should find the edge to accommodate everything. The problem is without live testing and not knowing your server details (like my.cnf file and many others) its really hard to make any smart suggestion, actually it could do more harm than good.
At least now you dont need to worry your Apache will crash if you do not restart every X hours, and that is good start.