Max_spare_servers set too high dynamically

SYSTEM INFORMATION
OS type and version Ubuntu 22.04
Webmin version 2.201
Virtualmin version 7.20.2
Webserver version Apache 2.4
Related packages php-fpm

Our vhosts are filling the memory on the server with old php-fpm processes, we believe this is due to the dynamic max_spare_servers setting.

With this set to 8 (as it seems to be dynamically), we are seeing very large consistent memory usage (~50GB) across as relatively small set of vhosts (~60).

Example pool file:

[164448877221794]
user = test
group = test
listen.owner = test
listen.group = test
listen.mode = 0660
listen = 127.0.0.1:8064
pm = dynamic
pm.max_children = 16
pm.start_servers = 2
pm.min_spare_servers = 1
pm.max_spare_servers = 8 ; <- Far too high
php_value[upload_tmp_dir] = /home/test/tmp
php_value[session.save_path] = /home/test/tmp
php_value[log_errors] = On
php_admin_value[upload_max_filesize] = 40M
php_admin_value[post_max_size] = 40M

These 8x child processes per vhost really add up and in most situations do not need to be there. Unless you are expecting very high levels of traffic you never really need 8x processes on standby, 1 or 2 is nearly always sufficient and more will be spun up as needed.

From the PHP-FPM doc comments:

;             pm.max_spare_servers - the maximum number of children in 'idle'
;                                    state (waiting to process). If the number
;                                    of 'idle' processes is greater than this
;                                    number then some children will be killed

If the default settings don’t fit
your needs edit them in either the php settings per domain or in the template for newly created domains. These settings are variable on use, for example I use what virtualmin sets however I don’t use any form of popular software which may or may not push fpm to it’s limits. I may not explained this well but there are always translation problems, forgive me if this is incorrect

Unfortunately I don’t believe this max_spare_servers value can be edited or customised because it’s dynamically generated based on the value of the max servers.

Correct me if I’m wrong.

Why not edit this value ?

I could, but I’d have to do that manually for every vhost and every one we create in the future, hardly a nice solution.

Also since it’s automatically generated theres a high likelihood of it getting overwritten in the future.

This is really a bug report that automatically calculating this value is not really suitable. It would be more sensible to be a small static value.

How is it determined. I’m getting this as default

pm = dynamic
pm.max_children = 11
pm.start_servers = 2
pm.min_spare_servers = 1
pm.max_spare_servers = 5

Not sure this classified as a Bug but a better setting for your system.

P.S. Ok worked it out, pm.max_spare_servers is basically pm.max_children divided by 2.

One site I found says pm.max_spare_servers should = pm.start_servers so you may have something here.
But other sites say different, who knows.

I think the advice in your screenshot is valid assuming you are running a single pool on the server, where you want to maximise throughput on a single site, I would say this is not typical for your average Virtualmin server.

It’s calculated here: virtualmin-gpl/php-lib.pl at master · virtualmin/virtualmin-gpl · GitHub

sub get_php_max_spare_servers
{
    my ($defchildren) = @_;
    my $defmaxspare = $defchildren <= 1 ? $defchildren :
        $defchildren >= 4 ? int($defchildren / 2) : 2;
    return int($defmaxspare);
}

I see Virtualmin has an issue tracker on Github, I will submit my suggestion there.

2 Likes

For those that stumble upon this thread, it’s possible to override this setting in the template:

@mattjones,

You can update the “Server Template” if you’d like to have each Virtual Server adjusted to your preferred setting.

*** These changes will apply to NEW Virtual Servers going forward ***

However, in order to affect this change on existing Virtual Servers, you’ll need to “disable” Apache and Apache SSL (or Nginx equiv) then re-enable it. When disabling, it simply deletes the server CONFIG not the files in your “public_html” folder.

The page to edit for PHP defaults would be:

System Settings > Server Templates > Default Settings > PHP Options

Then edit the field:

Additional PHP-FPM pool options

*** example ***

pm.max_spare_servers = 200
1 Like

You should be aware that memory use for processes like PHP-FPM is not a simple additive equation. Most of that will be shared across processes, as it is all identical pages of shared libraries and such.

That’s not to say that may not be an unnecessary amount of processes if this is a lightly loaded site, but also it’s useful to understand what’s actual unique usage and what’s shared.

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.