Virtualmin.com migration updates

News
1

Howdy all,

So, after a very, very, very long night, I’ve wrapped up our migration to Drupal. It turned out to be a little bit more than I could chew in a single sitting.

There are a few quirks in the new site, and I hope you’ll bear with us while I get them worked out.

A few known issues:

Forums and issues have weird “last updated” times, leading to the order of the forums and tickets being absolutely chaotic. The actual dates on the posts are correct (that took some work!), but Drupal keeps a separate last updated time table, and I haven’t figured out how to correct it yet. This will be fixed…in the meantime, search works better than on the old site, and new posts should naturally fall where they’re supposed to. So, if you can’t find your old topics, don’t fret, just start a new one. Once I’ve had a nap, I’ll dig back in.

The shop is possibly not quite working. We’ve taken this opportunity to switch to a new card processor that is more cost-effective, and also a lot nicer to work with than the old one. So, it works in test mode, but I don’t know if it works with real credit cards or PayPal orders. Don’t panic if something goes funny while you’re making a purchase. We’ll make sure everything works out. We’re also still waiting for the software repositories to finish syncing to the new system…so purchased products (assuming the shopping cart works) won’t show up immediately in the repo password database…you’ll have to wait a little while to install. These will be fixed by tonight (they’re at the top of my list, after a nap).

Documentation is still in a state of flux. We’re moving to a whole new organization, based on tasks rather than random blobs of docs spread across a bunch of files. It will be dramatically easier to find relevant documentation once we’re finished, but it’ll be another few days before everything has been merged and moved. In the meantime, there is a link to the old doc index, so you can still find everything that existed on the old site.

Lots of refinement is still needed, and I haven’t enabled a handful of additional modules we plan to turn on, but I felt like it was finally a better experience in almost every way than the old site, so it was worth a few bits of pain here and there while we work them out. Forums are finally sane (edit your posts!), and the ticket tracker doesn’t do crazy things with links (unless they were imported from the old one, in which case all bets are off). And, we can finally start adding new features to the site.

Anyway, I’m off to flip the switch, so you all can actually see this! Holler if anything is seriously broken and needs immediate attention.

Regards,
Joe

2

OK, so I forgot to mention about notifications. If you had notifications setup on the old site, they’re gone now (the notifications model just isn’t really transferable between Joomla and Drupal…dramatically different way of thinking of the problem). I haven’t yet turned on all of the notifications modules we’ll be enabling…so the options are a bit sparse, right now. You can follow threads by checking the subscriptions checkbox box, and you can make new notifications of nodes in the My Account page if you know the node. But, it’s pretty limited until I’ve installed and configured some additional modules.

Just wanted to reassure you that better notifications are coming very soon.

3

Howdy again,

So, I mostly got notifications working nicely now. You can setup notifications for content types (so, if you wanted to follow every forum, you could choose to get notification of all new forum posts), tags (so, if you wanted to only get forum posts from the News forum you’d go that route), or by a half dozen other less useful selection mechanisms (this will probably be simplified a bit soon, as it’s a bit intimidating at first glance).

I don’t actually know if notifications are actually working…we’re still in limbo on the mail server migration. It’ll happen tomorrow some time, but I think we have everything spinning for sending out mail, even though we don’t receive it on the new server, so it should work. You just can’t respond to it yet and expect it to go any place good (yes, I’m working on making emailed replies to forum posts work…it conflicts with the project issue tracker module, unfortunately, so those bugs need to be worked out, so it may or may not happen soon).

I’ve also published a new version of the download page. It was long due for an overhaul. I think it’ll make folks experiencing Virtualmin installation for the first time a little more calm about the process…and also a bit more clear about what they need to do to make a system Just Work with Virtualmin.

Lots of other little tweaks and fixes, including turning off the dastardly required preview “feature”. That was awful. I very nearly forget to actually save the comment every single time when the preview is enforced. But, it’s fixed now. Yay!

OK, back the trenches.

4

logged in and tried to submit support request - the form wants a ‘component’ but there are none (other than ‘none’) in the list which causes an error that sends the user back to the form to select from the non-existent list…can’t imagine how long your weekend was doing this change-over.

5

Hey Nick,

The “Component” thing bugs me too. We sponsored development of a “default” component, so it automatically selects one, but it still requires me to actually make sure there are components to choose for each project. So what was project were you filing a ticket for? I’ll make sure it gets at least one component.

6

Premature optimization is the root of all evil. So I haven’t prematurely optimized the site…and now that we’re seeing normal Monday load, we’re finding that Drupal is somewhat slower than Joomla in some cases (forum indexes are dramatically faster, though…they sometimes just didn’t work on the old site, timing out after 120 seconds).

So, I’ve been tweaking the MySQL database configuration a little bit, and learning how to look at a MySQL database to know what it’s doing. We’ll probably be seeing some downtime today for a minute or two at a time, while I restart things for the experiments.

Since problems always come in groups, one of our Xen instances (the Virtualmin demo) has begun to become needy and very demanding. It’s eating tons of resources for some reason (I suspect foul play…chickens, or some geese, maybe). Jamie is on the case, and will have it sorted soon, I’m sure. Looks like we’ll get to experiment with resource limits in Xen soon (Cloudmin has some capabilities in this direction, but they’re pretty new, so we’ll have a good reason to play with them).

Back to the salt mines!

7

OK, so I mixed up the IPs when migrating software.virtualmin.com. That probably contributed a little bit to our performance problems in Drupal, as Apache was serving virtualmin.com out to pretty much everybody for every request. This would have given 404s for install.sh among other things. Sorry!

That’s been fixed now (I think). And so the load on the CMS is a bit lower, and the download links should work again.

8

Joe,

No matter how many teething problems and temporary hitches there may be with this new version of the site, rest assured of one thing;

It’s just so much better than the old one already!

:slight_smile:

R.

9

Thanks for the reassurance, Rogi. It has been a bit more painful than I’d planned, and with a few more bumps in the road than I thought we’d hit. But, well…it is a major architectural change (Joomla and Drupal could not be more different; given that both are “CMS written in PHP”, the differences in underlying design are striking).

I’m fixing problems as fast as I find them, and I think in another day or two all the pains will be forgotten and all that will remain is a better site.

10

OK, all projects in the issue tracker should now have components, and all should have a default component. So, this problem should be gone. Holler if you find a project that doesn’t have any components. (All new projects will get default components…I somehow assumed that the default components would apply to existing projects, but they don’t. They’re like Server Templates in Virtualmin!)

11

OK, so the permission denied error has, after much gnashing of teeth, but knocked out.

You can now access your licenses again.

So I’m kinda dumb for taking so long to figure it out (note to self: Drupal modules cache hooks…so you have to disable and re-enable a module after making changes to hooks; not true of other code…just hooks).

12

I love it :slight_smile:

Very good work Joe

13

Awesome. Glad you like it. It’ll only get better from here. :wink:

14

So, I’ve added more docs (forgotten bits from the old site), and fixed a couple of broken links in the old documentation.

All CGIs should now be fixed (things like the script ratings, license checkers, etc.). The scripts.virtualmin.com website has been migrated (this is the fall-back download server, in the event a script, like Joomla, Drupal, Wordpress, etc. cannot be downloaded from its canonical source; it keeps copies of the latest version we know about on our own server).

The shopping cart seems to have an issue where it doesn’t create licenses. I don’t know why this is (it was working before I switched out of test mode), but I’m creating them manually in the meantime. So purchases can take a few hours to processes until I get that figured out.

15

Oh, and the performance problems have all been worked out. Jamie figured out why our Xen instances were being so hateful (disk I/O contention, strangely enough), and corrected it. So they are no longer trying to chew up the whole server.

I tuned MySQL a little bit, and enabled the APC PHP cache, and the site seems to be nice and snappy today.

16

Howdy all,

Some updates:

I figured out why orders in the shopping cart weren’t issuing licenses, and I believe I’ve fixed it (I can’t test further without doing a full order and sending money, and the terms of service at our payment processor only let us do that in test mode, which was already working). Life is hard sometimes. :wink:

We fixed a number of bugs in the reseller system (most folks don’t see that, but it was pretty broken, and a lot of folks get their licenses through that system, so we had to drop everything and get it straightened out).

Cloudmin download links were broken, and are now mostly fixed (at least for Linux, working on Solaris shortly). There also might be a problem with installs of Cloudmin from the new server, but I haven’t been able to track that down yet–the auth details seem to be getting put into the right places, but I still got an error report. So, that’s on my todo list.

I’m in the midst of fixing the missing API docs and bug submission from the Virtualmin support module; strangely enough, the same problem must be solved for both of them (adding a Drupal node from a Perl program), so once one is solved, the other will also naturally be mostly solved as well.

Back to the coal mines.

17

Imagine that… I have no licenses for any product under my account and yet I do.

18

Hmm what happened to the resellers VM login ? Seems that is broken as well.

19

Imagine that… I have no licenses for any product under my account and yet I do.

Are you still seeing no licenses?

20

Hmm what happened to the resellers VM login ? Seems that is broken as well.

It should currently be fully functional (to the best of my knowledge…there were many bugs remaining on Tuesday but we straightened them all out by last night when you posted). What problem are you seeing specifically?