'journalctl' task filling entire CPU core for minutes

SYSTEM INFORMATION
OS type and version Ubuntu Linux 22.04.5
Webmin version 2.202
Virtualmin version 7.20.2
Webserver version FPM-8.1.29 & FPM-8.3.11
Related packages journalctl / cron

Every couple of minutes, there is a job running that is completely filling one CPU core for a couple of minutes. In HTOP, I can see that the name of the job is:

journalctl -u postfix* -u dovecot* --since 7094778 seconds ago

The weird thing is; I completely ‘removed’ all mail related features in ‘Virtualmin > System Settings > Features and Plugins’ and also disabled them from booting in ‘Webmin > Bootup & Shutdown’ since the creation of this server. This means that postfix and dovecot logs were never accumulated in the first place.

So, why would this job take so long? And can I disable this job? If so, how would I do that?

Still after the recent update… Someone have any idea?

I guess you have disabled the cron job, even if disable services any cron jobs will continue to run. Go to webmin → system → scheduled cron jobs and check it’s disabled or deleted

One of these days this old dog will have to learn a new trick. One of the complaints when moving to systemd was that it was monolithic and not modular. I’m guessing that systemd has one huge axe log and is searching that. ( journalctl -u postfix*) It isn’t looking for the old style logs. It’s going through it’s one log and still looking as told. Hence the cron job disable comment? I believe @joe said that space allocation is dynamic so no telling how big the log file is?

I don’t see this in my cron jobs. What process would run it? Fail2ban? Seems like running the logs from the top every couple minutes would be excessive.

We have no idea on what the OP has set up, but the post seemed to indicate that some cron job was running. You are right though nothing in a standard setup would create & run a cron job

1 find out what on your system that is running email(an application PHP or Node or whatever you are running) - it doesn’t have to be Virtualmin and almost certainly isn’t else we would all be having problems.
cron jobs can be set up by almost any code
2.and you are not running AV on the system.

It is Virtualmin, but I thought we had it sorted out in the current version. @Ilia @Jamie why is this job still running on systems without mail? And, what’s it doing on a system without mail that could take so long?

1 Like

@scriptex You can solve your problem by running:

touch /var/log/mail.log

@Jamie, I think there’s a logical bug here. The code seems off because the hardcoded bw_maillog config option is tied to calling unconditionally get_mail_log() in both update_last_login_times() and add_historic_collected_info(). Can you please check?

Yeah we shouldn’t even be collecting mail stats if email is disabled. Fixed here : Don't collect mail stats if disabled https://forum.virtualmin.com/t/j… · virtualmin/virtualmin-gpl@288fad5 · GitHub

2 Likes

Thanks, @Jamie for the fix!

But what about this code snippet:

https://github.com/virtualmin/virtualmin-gpl/blob/master/feature-mail.pl/#L5672-L5682

Good point, I’ve checked in a fix for that : No need to check for IMAP or SMTP logins if email is globally disable… · virtualmin/virtualmin-gpl@91b4a51 · GitHub

I’m curious. Does journelctl offer up some sort of time stamp that allows a resume so you don’t have to start at the beginning every time?

If this does actually run every couple of minutes that would appear to be an excessive process.

@Jamie and @Ilia, thank you for the fixes!

I haven’t tested it yet. However, is there anything I need to do on my end after applying the future update?

Reason for asking:
The /var/log/mail.log file doesn’t exist on my server; it was likely never created because I haven’t used any mail-related services. So, why would the journalctl job take so long in the first place? There are no logs to analyze…

These two things are mutually exclusive. The journal is not individual log files. Modern systems do not write most things to individual text logs, they write to the journal, and you query the journal using journalctl (or other tools that are able to query the journal).

If you had a mail.log, the journal would not need to be read for this purpose.

But, if you don’t have mail, it’s just a bug that Virtualmin is trying to gather mail usage data, which is what the above patches fix.

@Joe, thanks for the detailed explanation!

I tried the quick fix suggested by Ilia (touch /var/log/mail.log), and it worked! My average CPU usage per hour has dropped by 15% since that.

Thanks, everyone, for the help!

This topic was automatically closed 8 days after the last reply. New replies are no longer allowed.