High CPU on reboot

SYSTEM INFORMATION
OS type and version CentOS 7
Virtualmin version 7.3
RAM 32GB
Swap 8GB
CPU Intel Xeon Silver 4210 CPU @ 2.20GHz, 4 cores

Hi,

I apologize in advance if this is not the right section as this might not be a Virtualmin specific problem.

We noticed that recent reboots are taking quite some time (est 20 mins) and uptime shows load of 20 to 30 times once the server is rebooted.

We have around 200 virtual servers on PHP 7.4, running on nginx. Server has 32GB RAM, swap disk of 8GB. 4 core CPU.

Most of the sites are running Wordpress. Opcache is enabled with opcache.validate_timestamps = 0.

From our checks, it seems each site’s php-fcgi is probably trying to load the opcache into memory and this could be the cause of high CPU. The load will go down drastically if we stop nginx or if we wait for around 30 mins (assuming somehow the cache is warmed up).

Another suspicion is due to slow disk I/O as we could see high %wa when running top command. Checking via ps auxf | grep " D" also shows most of it is related to the php process.

Is there a setting in Virtualmin for each site to load in sequence? We are considering to disable opcache totally but not sure if that would help. Increase CPU to 8 core is another consideration too.

One more thing to add, on a regular with low load, sometimes loading the Virtualmin dashboard will also cause a CPU spike. This is pretty random which we are not able to pinpoint the cause.

Wow 200, and I was worried about 10.

disk health? maybe a disk is failing?

disabling opcache in half the sites perhaps to check if that reduces boot time in half, could be a good test…
or moving to php-fpm with ondemand pm…

and i think you should increase to 8 cores for 200 sites anyway…

1 Like

We tried increasing the swap size and also increased CPU core to 8. It seems to help quite a bit.

Though at times we still face some random slowdowns / CPU spike while switching between Virtualmin/Webmin pages. That is probably a separate issue that we will need to check.

Increasing swap cannot improve performance. All it can do is prevent out of memory situations (i.e. OOM killer kicking in).

But, more cores can’t hurt.

But, I think your suspicion about disk performance is probably the most likely explanation.

1 Like