I use an Amazon S3 compatible bucket (at DigitalOcean) for Virtualmin nightly backups. Works great! I assume it’s using AWSCLI? I ask because I need to update the latter somehow. I believe Virtualmin automatically installed it when I initially configured S3 backups in Virtualmin.
The latest Python security update made me realize that my AWSCLI is out-of-date by quite a bit. How should I update AWSCLI manually, so as not to break the integration with the S3 Backup features in Virtualmin?
The latest AWSCLI is 1.29.15 (released today), whereas apparently I’m running AWSCLI 1.23.10 (released in May 2022). I only noticed owing to being unable to install the latest Python security update (4.9-2.el9) because of the old AWSCLI package:
Error:
Problem: problem with installed package awscli-1.23.10-2.el9.noarch
• package awscli-1.23.10-2.el9.noarch requires (python3.9dist(rsa) < 4.9 with python3.9dist(rsa) >= 3.1.2), but none of the providers can be installed
• cannot install both python3-rsa-4.9-2.el9.noarch and python3-rsa-4.8-1.el9.noarch
• cannot install the best update candidate for package python3-rsa-4.8-1.el9.noarch (try to add '--allowerasing' to command line to replace conflicting packages or '--skip-broken' to skip uninstallable packages or '--nobest' to use not only best candidate packages)
I want to make sure not to break backups, while also updating Python (particularly as this Python release is marked as a Security Update). Thanks for your help!
Hmm ok I thought I recalled Virtualmin saying it installed AWS-related CLI software when I initially setup the S3-compatible backups in Virtualmin. I’ve not installed anything manually on this server other than Virtualmin Pro (I originally set up my VPS using a Rocky Linux 9 disk image from Digital Ocean).
I guess Virtualmin’s S3 backups are using something else, so I don’t need to worry about disabling AWSCLI in order to update Python? I have run some Install Scripts using Virtualmin, that’s the only other stuff I’ve installed: Node on a couple websites, and Ghost on a couple others. I’m not aware of anything using something related to AWS, other than the S3-compatible nightly backups via Virtualmin.
S3 does use AWSCLI, so its does effect it. I couldn’t update python-rsa with using --best --allowerasing which did remove AWSCLI, that will break your S3 buckets so I wouldn’t advise that method.
We install many packages that aren’t our packages. I’m saying we don’t maintain that package and can’t fix that package. (Just as we don’t maintain the Apache, Postfix, MariaDB, SpamAssassin, etc. packages we use.)
We go to great lengths to use OS standard packages as much as possible; it’s a feature, not a bug. But, when upstream breaks their repos or packages, we’re at their mercy. (They have a lot more package maintainers than we do, though, so they’re much better at maintaining and updating packages than any small team likes ours could, in the general case.)
I didn’t say that. Virtualmin uses awscli for many AWS-related tasks.
Sorry I totally misunderstood, and assumed AWSCLI wasn’t in use at all by Virtualmin. Now it makes sense, along with my memory of Virtualmin mentioning installing it when I originally set up the S3-compatible backups around 10 months ago.
Sounds like I should research how severe the Python security vulnerability is, and likely consider trying manually to update AWSCLI while hopefully not breaking my Virtualmin S3 Backup functionality!
I had the same issue tonight updating my system. I issued this on my server:
repoquery -q --installed --whatrequires awscli
It responded with nothing. I looked in VM and it said awscli was installed around the time that I set up the server, and putting VM on there was one of the first things I did, so I have to conclude that it was installed at the same time as VM, as nothing else uses it.
I know you guys don’t maintain the thing, but was it installed when I installed VM originally?
I do not use AWS backups, I use Dropbox, so in theory I should be able to remove awscli without any repercussions. But I’d like to get more info on that before I start nuking things.
May I ask how Virtualmin originally installs AWSCLI in order to provide support for S3 compatible backups for the Backup and Restore module? I’m kind of guessing that Virtualmin originally used Amazon’s bundled installer described on the AWS page here (thanks to @stefan1959 for the link):
I ask how Virtualmin handles the initial install of AWSCLI, because I really don’t want to break or have to reconfigure my backups if at all possible (since I’m using DigitalOcean’s S3 compatible service rather than AWS, setting it up was difficult enough). My nightly backups are both vital and currently working perfectly!
I feel like there should be some way to update AWSCLI so as not to disturb my current S3 backup configuration?
@joe I realize Virtualmin doesn’t maintain or distribute this package, but I’m also surprised that with S3 backup functionality folded into Virtualmin, this particular package isn’t straightforward to update, so as to continue to have the Backup and Restore module working when other related packages (Python in this case) are updated via YUM?
Sorry for all the questions! I tried using YUM to update AWSCLI, just on the off-chance it’d been used by Virtualmin, but that doesn’t seem to be the case. And I can see that I don’t have PIP installed either. I’m just not sure how to update such that I keep my Virtualmin nightly backups working, while also applying the necessary Python security update?
No, we install everything that can be installed using the OS-provided packages by using the OS-provided packages. awscli on all RPM-based distros comes from the EPEL repos.
It was installed when you installed Virtualmin, as a soft dependency in the yum groups on RPM-based distros or the metapackage on deb-based distros.
Thanks @joe for the added clarity, I really appreciate it. I only ask questions here when I’m really at a loss for how things should be done to keep my Virtualmin Pro instance humming along!
So for those who are in the same position as I am here, I’d say it’s worth waiting for the moment, as from what I can tell, the upstream repos will be updated soon. I’m on an RPM-based system (Rocky 9.2), and I see in EPEL the old version of AWSCLI, and I also see that we’re not the only ones experiencing issues owing to the conflict between that old AWSCLI (1.23.10) and the latest Python-RSA (4.9): you’ll see related issues here that are causing problems sufficient to get the upstream repos updated.
Also I misspoke above, Python-RSA is different from Python itself. Python-RSA has to do with “encryption and decryption, signing and verifying signatures, and key generation.” The latest version of Python is 3.11, whereas the latest release of Python-RSA is 4.9 (with 4.10 currently in development). Note that Python-RSA 4.8 (which anyone reading this is probably running) was released 2021-11-24, and Python-RSA 4.9 (which currently isn’t able to be installed alongside our old version of AWSCLI) was released 2022-07-20 (one year ago, so hardly the most urgent of security updates, one hopes)!
Given all of this, I think waiting is better than trying to update AWSCLI with Amazon’s install bundler or PIP. Much as Amazon suggests correctly that their method of distribution will be the most up-to-date, I’d rather stay with OS-provided packages (via YUM), since those install in the proper directory with all associated dependencies. Once EPEL updates AWSCLI, we’ll be able to update both packages (AWSCLI and Python-RSA) in the usual way, without risk of breaking Virtualmin’s backup functionality.
p.s. Also thank you @stefan1959 for helping me research in the right directions here! I’m always trying to learn how to be better at this. I only run a small VPS for family, so I’m not too experienced
The pip install is quiet easy, the hard bit is the config. As long as you kept a record of the authentication and key and the bucket name you should be able to getting working. I had a play on a test machine (if you can do that you can test it) and its quiet confusing.
I use https://www.vultr.com/ for testing. Its a prepaid service with no contracts or min monthy spend.
Throw $10 in there and create a test machine.
A Cloud Compute with min specs if fine for VM at 7 cents for hour its well worth it.
Steve
The initial install is never the problem with installation using something other than the OS package manager. The problem is when you forget about it, and it goes without updates for months or years, because it isn’t being tracked in the same way as all your other stuff.
I go to great lengths to avoid installing things outside of a package manager.
