Hi, I am migrating from cPanel to virtualmin.
On same VPS specification, Virtualmin requires much more memory.
cPanel on centos7 running 76 accounts, with only 1 wordpress site per account, is only consuming 2.6GB of RAM,
while in Virtualmin almalinux running 18 development sites with little to none traffic I experienced mariadb service killed because of out of memory, and constantly consuming 3.3GB of RAM.
I think the only significant difference is only I created wordpress htaccess directives on apache install script instead of htaccess on virtualmin instance. Other configurations should relatively same.
Any thoughts why?
There are so many variables it’s silly to compare different environments.
The services that consume a lot of memory in a Virtualmin system are the same things that consume memory in a cPanel system. Virtualmin itself consumes about 100-150MB depending on configuration…or as little as ~15MB if caching is turned off. Every other thing using memory is literally not the control panel.
So:
Database configuration can make for wildly varying memory consumption. Have you made the database configuration identical on both the cPanel and Virtualmin system?
Apache concurrency model and PHP execution mode makes a big difference in how many processes and the size of those processes…but, not always an accurate reflection of real memory used (shared libraries can account for a lot of what looks like usage, but is shared). Are your applications running under the same execution mode in PHP on both systems? Is PHP configured with exactly the same libraries loaded? Did you install mod_php? That makes a huge difference in memory usage of Apache, even if you aren’t using mod_php to run your apps.
ClamAV? Are you running it on either system? It needs to be running on both or neither if you want a reasonable comparison. It takes more than 1GB of memory all by itself.
I don’t know what the source of the difference you’re seeing is, as you haven’t provided any information that can help determine what’s using memory. But…I have absolute confidence Virtualmin is not consuming a lot of memory (about 150MB is the upper bound, generally, maybe a bit more if you’re processing mail or if you have a lot control panel users at all times). I’m also reasonably confident cPanel/WHM isn’t dramatically smaller (might be bigger, I don’t know, I haven’t seen a cPanel system in 20 years).
If you have criticism of our default software stack and default configuration (which is what determines memory usage…not the control panel managing it), I’m happy to hear it. But, this is too vague to be actionable.
I noticed that a couple of people in the blue forum have started migrating their servers complete with paying clients on them from cPanel to VM in response to the forced theme change. One is using the GPL version, the other is paying for the pro version as a way to support the dev’s. You could ask those guys about performance, but my understanding is that cPanel is the biggest memory hog of all the panels so maybe something isn’t setup quite right?
No, this is not a criticism, this is only a result from observation, and I hope I can make it better usage on my virtualmin system by asking it here.
Can you help what information needed to be compared? I have upgraded the VPS to have bigger memory (thus having more CPU core) to keep it running, so comparing now might not an apple to apple comparison. However, memory usage still averaging on 3.6GB on upgraded VPS.
The thing that I have checked is only from htop command, and sorting it by memory allocation, it was dominated by mysql process at 5.5% memory usage on 4GB RAM followed by fail2ban.
I might not have enough knowledge on this, but this is what I think relevant.
WHM cat /etc/my.cnf:
# this is read by the standalone daemon and embedded servers
[server]
# this is only for the mysqld standalone daemon
[mysqld]
#
# * Galera-related settings
#
[galera]
# Mandatory settings
#wsrep_on=ON
#wsrep_provider=
#wsrep_cluster_address=
#binlog_format=row
#default_storage_engine=InnoDB
#innodb_autoinc_lock_mode=2
#
# Allow server to accept connections on all interfaces.
#
#bind-address=0.0.0.0
#
# Optional setting
#wsrep_slave_threads=1
#innodb_flush_log_at_trx_commit=0
# this is only for embedded server
[embedded]
# This group is only read by MariaDB servers, not by MySQL.
# If you use the same .cnf file for MySQL and MariaDB,
# you can put MariaDB-only options here
[mariadb]
# This group is only read by MariaDB-10.2 servers.
# If you use the same .cnf file for MariaDB of different versions,
# use this group for options that older servers don't understand
[mariadb-10.2]
virtualmin cat /etc/my.cnf.d/mariadb-server.cnf :
It was from virtualmin post install wizard: Huge system (2G or more) on which MariaDB is heavily used
All using PHP FPM.
WHM use mod_mpm_event if this is relevant
WHM installed php72, php73, php74, php80, php81
I’ll share result of dnf list | grep php | grep @ from WHM:
Did you try to replicate your whm my.cnf over to virtualmin my.cnf? (After changing, reboot mariadb).There are some entries listed that would suggest Virtualmin mariadb would use more memory due to the cnf file. On Virtualmin server is your db default-storage-engine=InnoDB?
Other apps installed like for like or do you have something installed that you didnt have on whm?
Also look if php fpm on the web for Pools and those settings , optimizing.
While every not variable but fixed value setting then is taking mem there. ( if some defaults then for all sites maybe… )
If you did choose huge mem for the Mariadb then this is also probable wrong if only 4GB system.
And log files also error logs / debug for apllication while after migration some maybe have to be configured other then defaults configs, version differences mariadb and so on, where there then mem leaks ( loops and so on) becasue of faulty settings.
No, I don’t really have skill to mess with my.cnf yet, so I try to keep it default if possible. If I want to configure my.cnf, I will modify it from virtualmin install wizard.
No, I use same other apps that is not cpanel and virtualmin default, like bitninja and rclone.
Sorry, I have hard time to understand what you want to say. But I think default php-fpm config looks okay. can somebody confirm if it is indeed okay for 8GB RAM with expected 200 or more users?
Now, htop shows 6.5G usage on 8GB RAM, which I don’t think is bad thing since there might be cache involved, but it shows that RAM consumption is from php FPM children.
I try to use smaller option for mariaDB configuration size on virtualmin’s install wizard, to 1G. After rebooting, it used significantly less memory.
Come to think at it, what is the meaning of size on mariaDB configuration size? there are small (256M) to huge (2G or more) size selection. Is that numbers represent mariadb memory usage limit? Or it is optimized for VPS with memory size (which I doubt, since @jotst2 said choosing 2G is bad idea on 4GB system)?
No. It is the terms the example config files that are shipped with mariadb use to describe them. We didn’t make the configs (we don’t have the expertise), the mariadb folks did.
Edit: Though, I see they’ve stopped shipping those example files, so I guess we’ll have to come up with another plan for helping users make reasonable decisions about config files. Presumably, though, the default config they ship is going to be appropriate for the average user.
But, again, you’re looking at the wrong danged thing. The database is not your problem (well, it became part of your problem when you told it to use way more memory than is appropriate, but your original 128MB, or even 2-3x that, config was fine, it’s when you started cranking it up to obscene numbers that it made no sense).
Anyway, your PHP processes look quite large and you have a lot of them. I don’t know what apps you’re running, but our WordPress system shows php-fpm processes with virt size of about 500MB and resident size of 83MB. That tells me your app is huge, or you’re loading a lot of libraries into your PHP. WordPress is not a small app and our website is not low traffic (it’s not high traffic, but it’s never idle). (Edit: I’ll also mention we’re running very close to stock configuration. I tuned the database lightly per WordPress docs about it, and PHP just has the libs that get installed by default and whatever WordPress needed. It’s very close to a base Virtualmin system.)
And, your database is actually quite a bit smaller than ours, in terms of memory usage. So, you may actually be starving your database of resources, causing PHP to wait longer than it should to answer requests leading to more and more PHP processes piling up, consuming more memory. You’ve gotta balance this stuff.
You need to dig in and figure out what’s actually going on, and we can’t tell you. htop/top tells you what to look at, not the root cause. I would look at PHP and your apps. You can enable slow query logging in the database to see if any specific queries in your apps are bogging things down, which would allow you to optimize for those queries (adding indexes, maybe, or optimizing the queries).
I am not an expert on PHP performance, so I don’t have solid advice for you, but your apps probably have tuning advice in their documentation.
Child processes are only forked when new requests will connect based on the pm.max_children and pm.process_idle_timeout which defines the number of seconds after which an idle process will be killed.
I’m trying this. As this article suggests that pm=ondemand is preferred on shared server, and cpanel use this as default.
How can I change the template for newly created virtual servers to use pm=ondemand? Preferably if it can be achieved from virtualmin GUI.
I can’t find PHP FPM template on virtualmin > system settings > server templates, and on other places.
Looks like they are old wordpress that not updated to recent version. Trying to update them now…
That’s because you’re looking in the wrong place. Go to the Virtual Server you want to change then Server Configuration / PHP Options and the mode selector is there.
Are you using a caching plugin? Some of those are notorious for creating huge memory issues if they’re configured improperly, especially if you’re using Cloudflare.
I’m looking for template for newly created virtual server, or configuration when I change PHP version. Since there are only 20 virtual servers, I have changed them all. But I think it should be an option to make all newly made to use pm=ondemand instead of pm=dynamic, so I asked above.
No, those who consume big RES mem are test sites that are not maintained. And there are no caching plugin on those virtual servers yet.
Note that pretty much everything about creation is configurable in Server Templates. The answer to “how to I make all new virtual servers get X option set with Y value” is “Server Templates”. (Account Plans cover user limits, but everything else is in Server Templates.)
