Virtualmin for Amazon EC2?

What’s the plan for VMPro on EC2? I saw a few VM2 references in the documentation and some questions in the forum, and also this: on the webmin site. But I would like a fairly simple statement/clarification of plans and goals.

For example: does EC2 even make sense! :slight_smile:

I say that because I’ve seen statements about EC2 not having data persistence between boots and other hints that EC2 just might not be aimed at “dedicated hosting”, but something somewhat different.

There are Virtualmin GPL and Virtualmin Professional users hosting exclusively on EC2. Several friends of ours who have new web startups are also using EC2 either along-side traditional hosting or exclusively. Many more are using S3 for storage of some or all of their content. It is a valid path. It is certainly economically feasible–for most users it is cheaper than a similarly equipped dedicated box, and about the same price as virtual private servers from other similar providers (like RimuHosting, Joyent, Gridzones, etc.). I wouldn’t compare them to vserver or OpenVZ or Virtuozzo based virtual servers, due to differences in the architecture and pretty significant reliability/performance differences, but I believe EC2 is still cost competitive with most major providers of such accounts.

VM2 is a UI layer to enable one to deal with many systems of many types (real, EC2, vserver, Zones, Xen). It makes large-scale hosting easier and more efficient. The target is mainly hosting providers, but if you aren’t a hosting provider but still have many hosts to manage, then it will serve you well (we use it a lot, and we only have three physical servers and a few virtualized servers–most of our testing infrastructure is moving onto Xen instances managed by VM2, and our demo is already running on a VM2 managed instance…and email goes through an EC2 server, so we’re eating our own dog food on all fronts).

As for the persistence issue: You get 10GB of persistent storage, and 160GB of ephemeral storage. The 160GB MUST be backed up to S3 periodically for safety, and it must be backed up whenever you shutdown the EC2 instance. This is a minor issue. You kinda have to view EC2 paired with S3 to consider it for hosting–you need the S3 storage layer in order to do anything really interesting.

One of the big sticking points has been resolved recently with the new IP address management that Amazon has announced, and Jamie has already added support for it in the next VM2 version. I don’t think Virtualmin needs to be aware of this change (though it used to need to be setup for Dynamic DNS based hosting, which is no longer necessary, though it isn’t a bad solution).

So, in short, we’re not moving everything to EC2 by any means, but I do think that for many classes of problem EC2 is a valid choice. It does have a reputation for reliability problems, which is definitely something you want to think about (a few hours of down-time may seem insignificant, and it probably is…but if it happens at just the wrong time, like when an investor or big customer happens to be looking into your business, it can have a pretty dramatic impact, so I consider reliability an important factor in host selection). S3 is a bit more proven and has a better reliability record (with one huge glaring many-hours long outage a few months ago that blows the whole curve, though). It’s certainly a good choice for backups. It’s off-site, fast, and they seem to have the data integrity problem solved very well (so once you put something into S3, I think you can pretty much consider the data “safe” forever and ever). Virtualmin Professional includes S3 backup support, by the way, though there are some limits–5GB per-domain, specifically.

Does that answer your questions?

There are problems with sending email from EC2 instances, such as messages being marked as spam.

Some people have apparently had some luck with configuring Postfix to use external SMTP services.

I’ve been playing with the Virtualmin AMI. I’d love to move all of our domains to Virtuamin / EC2, but the unreliable email thing is a big problem. PHP and CGI generated emails don’t appear to work at all.

You mention that email is through an EC2 server - I’d be interested to know how this was achieved?

I believe Jamie only uses the EC2 server for incoming mail, while his outgoing all goes through

That said, my understanding of one of the primary reasons for the new static IP feature was to make it possible to reliably send email from an EC2 instance. So, perhaps if you setup your EC2 with a stable IP address, and make sure all of your DNS ducks are in a row, you might find that things work fine. Obviously, I’d be interested to hear contrary evidence, as this would be a serious concern for many folks.

It isn’t really a static IP as such. The only change is the ability to reserve an address which will always be associated with your EC2 account. This is great for doing away with services such as DynDNS, and also means that if an instance did ever fail then a new one could be up and running within minutes and with no messing about with DNS

Other than having your address reserved, the system hasn’t changed. I believe that it is something to do with PTR records, which we have no control over.

Some people have suggested an external SMTP provider, but these all seem to be on a per-domain basis, and would be very costly for hosting many virtual domains.

I hope to find a solution that allows me to configure Postfix to use an external server, but to be honest I’m more of a designer than a developer.

One day, I dream of hosting all of my sites with Virtualmin / EC2, all with fully functional email and automatic nightly backups to Amazon S3. Then I can finally stop worrying about data loss and emails not getting through… and sleep peacefully at night!

So they don’t provide PTR resolution for their IP block? That’s just dumb. Has anybody mentioned this problem to them? :wink:

Does a paid AMI exist for the large instance? If so, how is it selected? Also, is it possible to "upgrade" from the small instance to a large instance on demand, without requiring a complete backup/restore?