Is Virtualmin multithreaded ?

Virtualmin
1

I’m doing a restore right now on a 7GB site from another server to a new server. I notice only one CPU at a time is handling this restore.

I know VM will use other functions on the server but for curiosity’s sake, is Virtualmin(GPL) multithreaded at all? Is any part of the restore process multithreaded?

It just seems like a waste that 7 out of 8 CPUs are doing absolutely nothing while this restore takes place. Decompressing a 7GB compressed file surely would benefit from 8 cores would it not?

2

Virtualmin is not doing the decompression and untarring. It is calling out to gzip/bzip2 and tar for those jobs. Those programs are not multi-threaded.

There was a recent discussion about pbzip2, which is a parallel implementation of bzip2, and Jamie added support for it as an compression option if you have it installed. I’m not sure if it went into 3.80 or will be in 3.81. This would resolve your use case of restoring a single large archive on a system with many CPUs.

In short, you’re actually asking us about software we didn’t write: tar, gzip, and bzip2. :wink:

Virtualmin, or rather Webmin, itself is a multi-process application, which spawns new processes (processes and threads in Linux are damned near equivalent in terms of time and resources) for each new task, and thus will take advantage of multiple CPUs. Many tasks are serialized for safety or simplicity (updates of multiple items and backups/restores of multiple virtual servers falls into this category).

3

thanks i expected VM was not doing the decompression

4

I found a compression utility called “pigz” that is supposedly a drop-in replacement for gzip. Are you familiar with it, do you see any problems using it, and do you think it is worthwhile installing it?

5

Howdy,

Unfortunately, I don’t know anything about pigz… my suggestion would be to either:

  • Just give it a shot, and see if it makes any large time or resource differences during the backup process

  • Wait until the next Virtualmin release, which would have built-in support for pbzip2 (you’ll see it in System Settings -> Virtualmin Configuration -> Backup and Restore -> Backup compression format). That would have both the good compression of bzip2, as well as the ability to take advantage of multiple processors/cores.

    -Eric

6

Hello from spain, i found a solution.

vim /usr/bin/gunzip

change the last line 'exec gzip -d “$@” ’ to 'exec pigz -d “$@” ’

:x

and enjoy restore multithreaded.