I got incremental backup on one server and everything works well except one big folder in one virtual server which is actually completely excluded in TAR incremental file under /var/webmin/modules/virtual-server/incremental - there are not recorded all backuped files.
That folder contains over 380 000 files - I don’t know if that is the reason why incremental backups are not recorded correctly but this particular folder is included in every single incremental backup - and it is over 120GB so there is issue with free space in this case.
If backing up from / and if by incremental you mean you have chosen the ‘Just add new files to archive’ option then you must also add -P as an extra command line parameter.
Without -P, files with / in them are compared to files in the existing archive with / stripped. Hence the comparison fails and you get files needlessly duplicated.
Actualy folder is under public_html - full path is ${domain}/public_html/data/uploads - and in other virtual servers it worksl flawlessly. They got same OSS CRM system installed and only in one case where folder is “too big” files are not written to “incremental log file” under /var/webmin/modules/virtual-server/incremental
OK thanks, not the same backup facility. I had, Webmin, System, Filesystem Backup in mind, which is an ancient backup system that can be used independently of Virtualmin servers.
I am still not clear what the problem is. You seem to be saying you have specified a large folder to not be included in incremental backup but it still is included in the incremental backup?
If you completely specify what is in the full backup and incremental backup schedule, what is in the logs and what is actually backed up it will be easier to understand.
I got one virutal server with large folder under public_html (full path is public_html/data/upload)
In incremental backups there is this folder included with ALL files even when there are only several files changed
This folder contains 380 000+ files
In other virtual servers where this folder is not so huge it is backuped in standard way meaning only newly created files are contained in incremental backup
This causes us a problem with space on disks because every single incremental backup is over 120GB
There should be only newly created files in incremental backup and there are all files in incremental backup for this particular virtual server
I’ve checked “tar incremental log file” under /var/webmin/modules/virtual-server/incremental - and incremental file for this virtual server is small (only 9MB - other big virtual servers has 160M size) - so there is not a full log of incremental files evidently
For some reason virtualmin is not writing full list of backuped files in full backup to this file which is than used for incremental comparison purposes
I’ve checked date (file creation date of files under public_html/data/upload) and they are old and unchanged
There is some problem in full backup writing of “incremental log file” which is than used for comparison and I don’t know why - my tip is because of large number of files in backuped folder - there is no other obvious reason for not creating right file for incremental tar comparison
Thanks. I see the run time took over two hours and was declared successful. Because of this it is difficult to attribute blame to Virtualmin for the mess the files got into.
Since the result is incorrect, I would be looking to see if there are multiple backup jobs running at the same time interfering with one another. You can check this with virtualmin, backups and restore, running backups.
How about trying this:
Check logs to see if backups are being activated more frequently than expected.
Disable all scheduled backups except one
If using complex schedule, carefully check only one minute is active and only one hour is active (not all hours). What does the complex schedule say?
Rent a cheap pay by the hour VPS and with a fresh install and minimal backups see if the problem is reproduced.
There are multiple different possibilities, such as a backup overlapping with what is being backed up and different virtual server backups overlapping due to symlinks
If you want Devs to look into it, you really need to fully specify a reproducible backup specification that occurs on a fresh installation. Backing up the virtualmin specification and providing it to Devs, or better still giving Devs access to a VPS with a reproduced result.
