creating a 'local repository' for offline install of virtualmin and its packages?

Searches of this forum, google/bing/yahoo, and ‘virtualmin --help’ for variations on “local repository offline install” have not identified any instructions on how to create a directory/repository of virtualmin installation modules that would allow a non-internet connected host to install virtualmin?

We have a client with a huge test environment that is isolated from the internet.

We are proposing the client move to using Virtualmin Pro to create and maintain their large collection of test domains and the pseudo users thereon.

We have already demonstrated how the virtualmin command line interface would allow them to construct and maintain the environment with a high degree of automation.

The client currently has a lot of expertise with Fedora/CentOS/RedHat, and so a ‘yum repository’-based solution is preferred.

We already have local copies of EPEL and other yum repositories for all of their infrastructure software; e.g., apache, postfix, php, mariadb, etc.

Right now our only solution with virtualmin would seem to be the re-engineering of where and how the ‘install.sh’ installation script gets its payloads from the 'net, and then installs them. (which seems rather brutal. there has got to be an easier/better way to do this)

Can someone point me at any documentation supporting offline, unattended installation of virtualmin? i.e., basically how to perform an ‘install.sh’ on a host that is not connected to the internet?

Thank you.

removed as duplicated somehow

Hi, I think it is not possible without talking to virtualmin guys. Anyway there are some great guides to create your own distro at debian website. have look at it. For offline installs I would create offline repo dvd or source of all in distro and then instruct apt or yum to use it while using install.sh - that seems logical from my point. Install.sh is just bash script installing and setting up virtualmin from programs in repos, so If you have all fedora repos mirrored, just setup your distro using those repos offline (from dvd or sftp) and run install.sh from virtualmin. That should work just fine. No re-engineering is need it (also thats nasty), in truth - install.sh is installing stuff from normal system repo and then download some few stuff from virtlualmin repo - server needs to be online so you are good… just use install.sh if you are pro user. you will always get safest and updated things… Also I dont think so it is advised to use reverse engineering and copying of pro virtualmin repos - violation of virtualmin tcs !! Be save there.

install on host that is not connected to internet is pointless to have - virtualmin offline would be like os without monitor and keyboard. if you want create vminstall.sh with your username and licence to automate script.

So the customer creates some 400+ domains with 50-10000 email users per domain, and you say that use of virtualmin is useless? I think that you are very confused and should stick to your areas of expertise - which seem rather limited. If you can’t answer the question, then you need not waste other people’s time in reading your asinine remarks, so please butt-out of my question.

My original inquiry stands.

.

I am no expert on this topic, and forgive me if i am out of my depth, however…
why could he not simply setup a virtualmin mirror? Locate that mirror on his own network (which may be both accessible from both LAN and WAN, configure his access to the client instance/s via ssh, and use his own mirror to perform the original install and updates?
If that mirror happened to be in the same location as the client existing machine…all the better. Of course i am making an assumption that the client build, whilst not accessible to the internet currently, is only configured this way and that it can be easily reconfigured to be internet accessible. If not, then i too am going to place a negative on this idea…what machine of the nature that you are talking about would never be purchased with zero intention of making it WAN capable?

It sounds to me as if what should have actually happened is that a cloud server should have been deployed for this testing phase rather than a static box sitting in a room in a client office somewhere. It surprises me that a pro would even consider such a test setup without any internet capability at hand!

I dont mean to offend in any way…please take this as some constructive criticism from a person who is a relative newbie at this, however, i have been in business for a long time and from a business point of view what i am saying is definately not newbie.

what machine of the nature that you are talking about would never be purchased with zero intention of making it WAN capable?

It’s a test LAN for stochastic simulations of intranet traffic and monitoring thereof…

You can run Virtualmin unattended by using the --force flag to install.sh. You need to make sure your system is “ready” for Virtualmin if you do this though, as it’ll cause it to skip several tests that help prevent broken installations, like the FQDN test and the test for sufficient memory (I don’t remember what it’ll do if there’s not enough memory; it would either have to fail, or create a swap file without asking, and I can’t remember which decision I made on that for a forced installation). This is somewhat less well-tested than with the old installer, so if you run into any problems, let me know.

You can get usage information for the install script by running it with the --help flag.

You have a few options for locally hosting Virtualmin files, though there’s no good way to handle Pro installations without network access.

You could set up a local caching proxy (like Squid) to automatically “mirror” the data without having to maintain anything. That would be how I’d handle it. You could also use wget with its recursive download feature to fetch the entire repository. There is nothing magic about our repos (except the authentication for the Pro repos); we mirror them using rsync, as it’s all just files.

It looks like you currently only have one Pro license so that probably eliminates Pro authentication as a problem to be solved here. The GPL repos are cacheable and copy-able, using any standard tools you like for the task. If you end up deploying a bunch of Virtualmin Pro instances and you need to be able to do it without internet access, you can get in touch with me and we can sort out how we can make that work for you.

Thank you.

It’s not the easy answer that we hoped to hear, but it is manageable.

Ultimately it’s “the magic” of the ‘virtualmin’ command line interface (CLI) that will make administration of their entire test environment possible; e.g., being able to reconfigure their entire network with a single command like “virtualmin modify-dns --all-domains --ttl 300” will be infinitely better than their current situation.

And since we can’t guarantee that a caching proxy can preserve a configuration snapshot for months after-the-fact, we’ll look into using “wget with its recursive download feature to fetch the entire repository” for use with a non-Pro Virtualmin license.

“It’s not the easy answer that we hoped to hear, but it is manageable.”

It’s not? What were you hoping for instead?

And since we can’t guarantee that a caching proxy can preserve a configuration snapshot for months after-the-fact, we’ll look into using “wget with its recursive download feature to fetch the entire repository” for use with a non-Pro Virtualmin license.

What do you mean by “configuration snapshot”? It’s possible to replicate a Virtualmin configuration just by copying the /etc/webmin directory from a golden master system (one without domains that otherwise is configured the way you want). But, if you mean a specific version of all packages, I’d discourage locking versions (though it’d be possible to do by not updating your mirror), due to security concerns. Webmin/Virtualmin very rarely has security bugs, but when we do, it’s important to be able to get them out to all of your systems quickly.

It would be great to share all of the details with you, but not in a completely public environment.

Right now this customer creates and maintains several hundred named domains that are not “public” on the internet, but exist on a intranet having very specific needs and requirements.

The environment is such that the administrators don’t have the luxury of being able to “yum update” and then blindly accept everything that happens to come in from the internet. All software installs and updates are done with/from a handful of gated, static repositories that are themselves only changed with great caution and after great consideration.

Perhaps the best analogy that can be offered: Consider a fleet of vehicles (e.g., cars, airplanes, tanks, etc.), where each physical vehicle has its own named domain, with open ports, a firewall, etc. etc. …

OK, that makes sense. If they’re private, it’s reasonable to roll updates selectively.

There’s a couple of ways to do the client-side of things.

You can either point software.virtualmin.com to your own mirror server(s) in your local DNS, or you can rebuild the virtualmin-release package (if using CentOS/RHEL) to include your repo server address and modify install.sh to point to the mirror (and the Debian/Ubuntu repo address is set in install.sh). If it’d be helpful, I can add a command to select download server during installation…that still wouldn’t handle the virtualmin-release package creating repos pointing to our servers, though. The virtualmin-release rpm files are here, if you want to do something with that: https://github.com/virtualmin/virtualmin-release

If these servers are identical except in name/IP, you might also just want to make an image with Virtualmin already installed and replicate it that way. You’d need to update the hostname, and a few other bits, but you have to do that anyway.

Understood.

A command or option to specify “download server” would have uses beyond just the initial installation.

Within a “family” everyone starts out as an “identical clone”, but due to operations and ad-hoc updates will quickly become a unique individual.

Currently there would be at least several dozen families, with many more dozens expected to exist in no time; especially if things can be managed with a consistent interface, like the Virtualmin CLI.

Now, if there is a “Virtualmin Developer’s Guide for Writing and Implementing new Modules” (e.g., for things as complicated as Apache and Bind, if such modules were not already available with both GPL and Pro), then we could probably manage the family differences via a custom module that installs and plugs-in to Virtualmin, and which is then used to further customize an entirely generic individual when it is created (i.e., when creating a new virtual server from a server template)…

This is a good story to tell, and for us to try and sell to the customer…

Thank you.

There are docs for developing Webmin modules and Virtualmin plugins (which is a special case of Webmin module with some hooks to allow customization at various stages in the Virtualmin domain creation/update process). You can also use the pre- and post- hooks in Virtualmin for simple customizations. Depending on what you’re trying to do, you may only need a couple lines of shell scripting, or a custom Server Template, or a custom skel/ directory. But, you can also create modules, if you know some Perl.

Webmin developer docs are here: https://doxfer.webmin.com/Webmin/Development

Virtualmin plugin development here: https://www.virtualmin.com/documentation/developer/plugins

And, the new virtualmin config-system config management tool is available and extensible, as well (this is used during installation, and can be extended to perform custom installation stuff, if you’re doing things more complicated than a simple shell script or whatever might do): https://github.com/virtualmin/Virtualmin-Config

So, it really depends on what you’re trying to accomplish as to where you’d want to plug in your custom config or functionality. I’m happy to answer questions and point you in the right direction on how I’d implement something. It’s probably easier than you think, if you haven’t thoroughly poked around in Virtualmin yet; there’s a lot of really deep customizability that we built way back when we had mostly really hardcore power users, who wanted to be able to customize everything.