Upgrading Ghost Install Script from 5.26.3 to 5.40.2 results in 503 Error

SYSTEM INFORMATION
OS type and version Rocky Linux 9.1
Virtualmin version 7.5 Pro

The other day I installed Ghost 5.26.3 on a subdomain using Virtualmin Pro’s Install Scripts feature. It worked great, I set up a few pages on my new Ghost website with it.

Today I noticed there was an update to the Ghost Install Script version, so I dutifully clicked the Upgrade button in Virtualmin to upgrade to Ghost 5.40.2.

Now though, when I go to visit my Ghost website, I simply get a 503 Service Unavailable message. I tried restarting Ghost in Virtualmin’s Install Script area, to no avail. I SFTP’d in and looked at ghost.err but that file is blank. I peeked into ghost.log and it just had the original welcome message with no errors shown.

How might I go about getting my Ghost website back online after the update? I’m not sure quite how to troubleshoot, since the Install Scripts usually handle upgrades smoothly.

A quick note to add that according to Virtualmin Pro’s Install Scripts view of Ghost for my subdomain, it shows Ghost’s “server status” as “running”. Restarting the script here also shows Ghost as stopping and starting, without error.

But when I look in Webmin > System > Running Processes I don’t see Ghost running on this subdomain (though the older version of Ghost as an Install Script is running on a couple other domains that I haven’t yet upgraded).

maybe someone on @staff can help?

Hello,

Thanks for pointing this out. It’s a bug, because Nodejs also needs to be upgraded for Ghost 5.40.2 to work. The fix should be is to upgrade Nodejs and all dependencies to use version 18.15.0.

We will fix the script to disallow upgrades from Virtualmin. Ghost is extremely fast developing software.

Ghost 5.40.2 definitely supports Node 16 (I’m using Node 16.2.0 locally with Ghost 5.40.2).

Ghost’s Install Script is the reason I upgraded this particular server of mine to Virtualmin Pro.

What is the suggested upgrade path for Ghost via Install Scripts, if not clicking the Upgrade button? Node was installed by the Ghost Install Script originally, correct?

We download Nodejs to install Ghost.

If you run systemctl status ghost-example.com-3000.service on this process, what does it output?

We actually run the following command to install updates:

~/ghostblog/node_modules/.bin/ghost update N.NN.N --no-setup-linux-user

But here is the thing. While we were chatting, Ghost 5.41.0 was released, with which the local upgrades started working again. I’m sorry, I don’t know what the issue was.

I will look into unlocking further upgrades using Virtualmin.

I think it’s helpful to compare two Ghost installations on my server. One is functional (not yet upgraded), and the other is identical other than that I clicked the Upgrade button for the Install Script:

$ sudo systemctl status ghost-beemuse.com-3000.service
● ghost-beemuse.com-3000.service - Start Ghost server for beemuse.com (Virtualmin)
     Loaded: loaded (/usr/lib/systemd/system/ghost-beemuse.com-3000.service; enabled; vendor preset: disabled)
     Active: active (running) since Fri 2023-03-03 06:54:30 PST; 4 weeks 0 days ago
   Main PID: 1357 (ghost run)
      Tasks: 22 (limit: 22944)
     Memory: 262.0M
        CPU: 14min 7.146s
     CGroup: /system.slice/ghost-beemuse.com-3000.service
             ├─1357 "ghost run"
             └─1937 /home/beemuse/public_html/bin/node current/index.js

$ sudo systemctl status ghost-buzz.beemuse.com-3002.service
Unit ghost-buzz.beemuse.com-3002.service could not be found.

Oh, wow. What is the output of systemctl | grep -i ghost command?

I will double check Ghost Virtualmin install script code to see if there is some way for the service file to be delete upon upgrade.

$ sudo systemctl | grep -i g
ghost-beemuse.com-3000.service   loaded active running   Start Ghost server for beemuse.com (Virtualmin)
ghost-redacted.com-3001.service   loaded active running   Start Ghost server for redacted.com (Virtualmin)

The above shows my expected two Ghost 5.26.3 servers running.

The third Ghost instance that was running on a subdomain (on port 3002) right up until I clicked the Upgrade button to Ghost 5.40.2 is no longer appearing here as a service, despite being reported as Running by Virtualmin.

Of note, when I clicked Upgrade in Virtualmin, it not only reported no errors, but showed the service as stopping and starting properly. And when I click Restart in Virtualmin’s Install Scripts (on the upgraded one that’s not working), it reports this:

Shutting down the server process for Ghost ..
.. done

Starting up the server process for Ghost ..
.. done

But rechecking with sudo systemctl | grep -I g still shows no such service running.

I just realized I missed one of your replies above about version Ghost 5.41.0

At worst, I can simply delete this Ghost instance, I’d set it up for staging… thankfully! It’d be really cool to have a safe upgrade path for the two that are still running on 5.26.3 though, via Virtualmin’s Install Scripts.

I have double checked and just released (30 minutes ago) Ghost 5.41.0 doesn’t have this bug on upgrade, even when changing Nodejs versions.

Try to manually upgrade Ghost to unsupported version using this page:

However, it will not restore deleted service file. You should copy it, and then tweak it manually using the working as example. When done, run systemctl daemon-reload. I’m not sure what made it disappear, however, if the service file is deleted, it shouldn’t be showing Running. I will check to fix that.

Here’s something interesting. So I manually upgraded Ghost from 5.40.2 to 5.41.0 just now, as you suggested:

Then I went to check on the service file. And I found this:

ghost-beemuse.com-3000.service
ghost-beemuse.com-3002.service
ghost-redacted.com-3001.service

So the service file for this Ghost instance wasn’t deleted, it just doesn’t reference the subdomain as I’d thought. Here’s the output for that status of the one on port 3002:

$ sudo systemctl status ghost-beemuse.com-3002.service
○ ghost-beemuse.com-3002.service - Start Ghost server for beemuse.com (Virtualmin)
     Loaded: loaded (/usr/lib/systemd/system/ghost-beemuse.com-3002.service; enabled; vendor preset: disabled)
     Active: inactive (dead) since Thu 2023-03-30 16:26:24 PDT; 17h ago
   Duration: 2d 46min 17.033s
    Process: 2631936 ExecStart=/home/beemuse/domains/buzz.beemuse.com/public_html/node_modules/.bin/ghost start --no-setup-linux-user (code=exited, sta>
    Process: 2831884 ExecStop=/home/beemuse/domains/buzz.beemuse.com/public_html/node_modules/.bin/ghost stop (code=exited, status=0/SUCCESS)
   Main PID: 2631976 (code=exited, status=0/SUCCESS)
        CPU: 1min 3.482s

Although it still shows as running properly after the upgrade here:

Then the only reasonable explanation to this is that Ghost restarted itself (outside of systemd) after the upgrade.

Does the Ghost instance work if you visit the website with this particular Ghost install?

Visiting the website still shows 503 Service Unavailable (that started happening after upgrading from 5.26.3 to 5.40.2, and is still the case after upgrading to 5.41.0 from 5.40.2).

And the output above shows it as inactive (dead) since that first upgrade attempt, regardless of showing as Running in Virtualmin?

Have you tried manually restarting the service using console?

Ah, it’s back up! I tried clicking Restart (of the Ghost script) in Virtualmin during the first failed upgrade attempt, but not after this latest upgrade to 5.41.0.

I just ran sudo systemctl restart ghost-beemuse.com-3002.service and you’re right, it all pulled up again :slight_smile:

So for the two live sites currently running on 5.26.3, I should be able simply to upgrade “to unsupported version” 5.41.0 and not have these troubles, I’m hoping? Thanks again for your time on this and quick responses :slight_smile:

Dealing with Nodejs dependencies can be frustrating. It can take time.

I’m glad that it’s working for you though!

I will see what could cause this issue with showing false-positive running state.

Yes, try for now to upgrade manually to 5.41.0. It should work. I will push the update to Ghost script a bit later today.