Hi,
after reading the documentation, I’m still struggling to figure out how to configure a scheduled backup correctly.
What I want is:
A full backup on every Sunday
Incremental backups Monday - Saturday
After a successful full backup on Sunday, all older backups (full + incremental) shall be deleted
What should I put where in the delete settings?
Should the full backup schedule delete backups older than 1 days, and it will delete all backups prior to that, including full and incremental? And the incremental backup should have no delete setting?
Or what should it be?
The destination file should be the same for both schedules, like /var/backup-drive/all-servers-%Y-%m-%d?
amazing yes! but at least using the latest version.
I think even the documentation has improved in that time, not to mention the number of posts on the matter in the forum.
I guess if I could find the spare time to type with my one digit I could put a set of instructions together
Yes I know - I was not being ageist. just trying to illustrate that much has changed in the last few years I have been using Virtualmin - I guess we are all older
Jamie and I have been working on it about half of our lives, too. (Actually more than half for Webmin.)
We’re glad you’re still around! And, I take a little satisfaction in the fact that I have painstakingly kept every forum post since the very beginning of having a Virtualmin forum, through so many forum software migrations I can’t remember them all…at least five migrations. It’s fun to see how long folks have been around, even if we haven’t seen you very often.
And, actually, this is an area of the UI that could probably use improvement, though how to improve it without removing functionality isn’t jumping out at me. I think backups are one of those things that you setup once and never think about again…so the pain doesn’t get complained about much.
I think that would work, at least as long as you’re not backing up more than once a day, though it might not be super intuitive to find the full and incremental (differential) backup.
I’ve usually made a separate directory for full and incremental backups, and I keep full backups longer than incremental for long-term archival purposes…like, “Oh, I need to dig up some detail that got changed six months ago, but I don’t need the exact day, just from before things got broken or before I deleted the email or file.” or whatever, and only keep incremental backups for one full backup cycle (a week in your example).
But, for your preference of never having more than a week of backups and only have a day of backup after Sunday (which very feels scary, to me, that’s barely a backup at all), I think you’re on the right track. Putting them in the same directory and allowing full backup schedule to delete anything older than the current backup should lead to all incrementals from the week being deleted.
I like to keep full backups for at least one cycle+1 (though actually much longer), so you always have two full backups. But, you can certainly decide to only keep one, but a full backup will not delete incremental backups.
If the backup types are stored in different directories, you can manage the deletions independently. I think if they’re in the same directory and with exactly the same filename format, the full backup can delete incremental backups. (Though I don’t think I’ve ever done that, so I’m not actually sure…the docs hint that it will simply find all older files of the right format in that directory and delete them, though.)
The backup and restore docs cover most of this. I think we have better docs than 16 years ago, or at least we certainly have more docs than 16 years ago.
Backups are such an important part of any management structure.
sadly all too often ignored or set aside for some other insignificant person to deal with
they should be the first consideration of any IT manager as part of the operation’s Disaster Recovery Plan. - what if you are run over by the proverbial bus tomorrow?
dust it off regularly and check it still works and that whoever will need it, can still use it.
Best thing I’ve every done was setup schedule backups useing Virtualmin backups with AWS, at around $3 per month well worth it.
And from what I’ve tried OS independent.
Any S3-compatible API should also work, AFAIK. There are several options that are a little cheaper for the backup use case (based on how they bill…if it’s cheap to get data in and cheap to store data month to month, then it’ll be cheap for backups, since it’s not being downloaded ever again, unless something disastrous happens).
It’s not super obvious in the GUI that that’s the case, but you don’t have to use Amazon, and most storage/cloud providers offer an S3-compatible API now.
I do a full backup everyday and differential every four hours so I was happy with that price. Better then $10 per month backup at vultr although they do a full system backup.
I do have a cloudflare account so I might try them for a month and see how it compares.
I tested the backup with Cloudflare R2 and Backblaze B2 both today and it is working perfectly.
One important thing is to create API for all buckets so it can create additional buckets with proper region wnam, enam, weur, eeur, apac, oc, auto.
if a bucket name is hello-bucket, you can use path like hello-bucket/backups/virtualmin/backup-%Y-%m-%d or simply hello-bucket/backup-%Y-%m-%d
both options are reliable and perfect. cloudflare integration with their services is awesome. B2 is much affaordable than R2 but both are great as per different use case.
I have now created a Python script to remove my old backups. My backup storage is mounted via NFS, so it behaves like a local directory.
In case anyone is interested, I’ve uploaded it to Github. To install, upload the purge_virtualmin_backups.py and purge_backups.py to /usr/local/sbin and grant execute permission.
It can be configured as cron job in Virtualmin and should be executed after a full backup. It verifies that the last Virtualmin backup was successful and will then remove all old backups.
The cron command is: /usr/local/sbin/purge_virtualmin_backups.py /path/to/backup
My backups purge after 4 days, is it the date format that causing your issue.
Also my only concern with this, if you deleting all old backups is the last one may not have the data you want.
Like a customer wants a backup from 2 days ago as one day the website was infected.
I have 2 backup jobs, a full backup on Sundays, and incremental the other days. It was obscure to me, where to configure which delete, to achieve my goal.
Basically, my backups should provide recovery when something goes wrong with the hosting system. If a customer requires a version history of his website, I’d setup a dedicated backup job for that, excluding the ton of emails and other data.
the amount of backups to keep is indicated by the top arrow and the second arrow indicates that you don’t need a cron job as you can execute a command after the backup has been completed, I have been using this method for over a decade and to be fair has never failed
Like jimr1 screenshot, using delete after 7 days would do your full backup.
The incremental prob the same, that backs up from the last full (not last incremental).
