@Jamie, another thought! What happens to the Webmin cache is such case scenario? I mean, the stats server internally uses quite a lot of the Webmin APIs, which in turn use different functions that may involve caching.
Could that be the culprit of the problem, you think?
Do you have any idea what else could cause the size of stats.pl to grow? I have checked everything multiple times, and I could only find one potential issue, which I believe I fixed in the patch above.
@Jamie, I have dug in more and realized that calling the Webmin API every second is costly and unnecessary, as well as calling eval "use JSON::PP"; on every call to encode/decode JSON. Additionally, it was inconsistent when JSON::XS was installed, as we should have used JSON::XS in that case.
Deployed on test server, mem usage for stats.pl immediately went from 200 to 92mb used. I’m going to leave it like this until morning (GMT) and report back if it’s ok.
Sorry for no responses, my friday took several unexpected turns.
I was very happy to read about your discoveries regarding webmin api and json encode/decode. I’m glad this unfortunate and obscure bug produced a robust patch that even improved efficiency and consistency. Thank you
It seems to me that with the last patch (and also when trying the first patch before) the CPU dial is not moving: staying on 0 when logging in and on 80% or 90% after reboot or a shutdown. There also seem to be a considerable delay in drawing and updating the Stats History graphs, that sometimes are not drawn at all, but that seem to be a change that came with the 21.20.3 version.
What i mean is merely the display: the CPU dial on top of the webmin page seems not to move at all, being stuck on a certain value. The Stats History graphics also sometimes never appear for me, even if I wait several minutes, and if it does, it stops moving frequently for several seconds. I think I noticed those things only after logging out and in of webmin or rebooting.
The stats are supposed to stop for a second or two during collection time. It purely depends on your overall system’s performance. What are your system’s specs?
1GB RAM, 2GB SWAP, 2 vCPUs and 40 GB SSD. I only serve two sites with very low activity, so the resources I use are very low.
When I refresh the display using that round arrow on the right, the dials are moving for a moment, but then stop moving altogether. The graphs also does not appear. I think it came about only after logging out of webmin and in again.
Just checked the test server and indeed the memory usage issue is still there.
I’m getting dials and graphs, but I notice the dials are no longer moving smoothly all the time. Cpu dial starts filling up smoothly as usual but shortly after starts jumping around from one value to the next until settling on 0-1% at idle. The erratic behavior doesn’t show abnormal usage, it’s all perfectly within what I see every time I open up virtualmin as it fetches the stats, but the display of values begins stuttering after a while.
Our test server is available for you if you need/want it to troubleshoot the issue.