Well, nothing changed. Some vhosts still fail from time to time. And it is wierd that “small” backups fail as even as “large” backups do. Nothing to do with the size I presume.
Yea, I’m the same. I’ve noticed however that disabling the backups completely results in webin not crashing once a week as was previously the case. This is clearly some kind of resources issue - at least on my machine.
I found this post on the aws forums
https://forums.aws.amazon.com/thread.jspa?messageID=584023
I moved my buckets from “Ireland” to “US standard” 3 days ago and since then I’ve not had one failure.
Ironically I’m actually based in Ireland but interestingly I also use Hetzner in Germany to host. However one virtualmin box always backs up fine to Ireland while the other (almost identical: more vhosts) consistently has problems doing the same. I’m seriously tempted to move buckets though after reading that
Don’t bother. I’ve always been on US buckets. Same problem.
Ha! @eddieb
Thanks for responding so quickly; I was poring over the following AWS CLI command when your reply came in:
aws s3 sync s3://oldbucket s3://newbucket --source-region us-west-1 --region us-west-2
…I don’t think I will bother now.
A few months after the last post, I’m getting this exact issue.
Did anyone got it solved?
Thanks.
I migrated my most problematic box to entirely new hardware thinking this was fundamentally a hardware resources issue and it has made little difference. If anything the issue has got worse because it also periodically crashes Webmin itself.
wrong place for reply
Eric, does it use the s3cmd library? It recently got updated but the failures persisted.
Alternatively, this doesn’t solve the problem for everyone, but enabling Google Cloud Storage backups would be nice.
Well, I switched away from AWS to Rackpace. Works like a charm.
No more failure for more than two weeks :)))
I added Rackspace Cloud Backup yesterday so only have 1 day of Full and Inc backups, but they all worked fine. The last Full backup with S3 only backed up one site, so its still unreliable.
So I will use Rackspace from now on.
The only thing I dont like is Rackspace splitting the files up, adding .00000000 etc at the end. This post states there is a bug restoring them, but looks like it will be fixed in the next release https://www.virtualmin.com/node/27253
I’ve stuck with the Amazon S3 backups and I’ve noticed that since the recent upgrades which utilise Amazon’s AWS the old errors occur rarely now. This is progress, however what’s happening now is that Webmin/virtualmin itself crashes and occasionally MYSQL tables crash on sites that have been backed up. This makes backing up like a mini-drama seeing what carnage is generated!
Can I ask anyone who is successfully backing up to amazon to share their chunk size setting? (Backup & Restore / Cloud Storage Providers / Amazon / )
Way back when this problem occurred, we moved to a 100MB chuck size and the problem completely disappeared. The only other change we made was to a core file referenced in another post – I think it increased the “retries”, but I’d have to dig that information up, and that file may have been overwritten by the latest updates anyway. Our 100MB chuck size persists and seems to work 100% of the time.
Settings are found under: Backup & Restore / Cloud Storage Providers / Amazon /
i moved to 100MB chuck size , it works find but some backups are over 100MB .
i have tried to set 200MB, it still failed. any ideas?