Firewalld 100% CPU after a couple hours

Virtualmin
1
SYSTEM INFORMATION
OS type and version AlmaLinux 8.10
Webmin version 2.303
Usermin version 2.203
Virtualmin version 7.30.8
Theme version 23.03
Package updates All installed packages are up to date

After reboot and tracking cpu usage using top firewalld uses around 20% CPU but it increases exponentially until it freezes the whole server after sometime after reaching 100%.

logs
logs1270×689 232 KB

top
top737×483 145 KB

this top report was one hour after reboot
Logs look normal to me. Anyone has any idea what might be causing it?

2

My first impression is ‘circular firing squad’. :wink:

I’d try,
Stop fail2ban.
Restart firewalld.
Start fail2ban.

The repeated ‘already enabled’ makes me think firewalld and fail2ban are out of sync. Long story but firewalld must start before fail2ban. If firewalld got restarted that has been a problem in the past. There is a fix in the firewalld config to keep the fail2ban stuff during restart but it is now disabled by default. I think people restart the firewall wanting a clean/fresh restart. This isn’t good when you have other programs relying on it though.

EDIT : Also, a large swathe of 138.199.x.x appears in that screen shot. That could be a large scale DDOS? As a quick test you might add 138.199.0.0/16 as a temp block.

3

Thank you for your help, i’ve done both, resynced fail2ban and firewalld and added that ip range as a temp block to see if it works. I’ll keep watching and update with the results.

1 Like
4

I suspect that means fail2ban is messing up the rules. It shouldn’t be seeing traffic from the same IP once a rule has been created.

5

EDIT : Also, a large swathe of 138.199.x.x appears in that screen shot. That could be a large scale DDOS? As a quick test you might add 138.199.0.0/16 as a temp block.

Blocking the IP Range solved the cpu usage issue, thank you!

I suspect that means fail2ban is messing up the rules. It shouldn’t be seeing traffic from the same IP once a rule has been created.

They show up once for port smtp, 465 and submission, but when blocked they dont show again, i think this 3 ports are the same arent they?

my jail.local its like this for example:


enabled  = true
bantime = 12d
bantime.increment = true
bantime.factor = 1
bantime.multipliers = 1 24 84 720 1000
findtime = 1d
bantime.maxtime = 2d
port = smtp,25,submission,587,465
action = %(action_)s %(action_abuseipdb)s[abuseipdb_category="11,18",matches="spam/brute force attack blocked attempt from fail2ban"]```
6

The reason I said to do a temp ban is that’s one hell of a lot of addresses to block. Seems to be Amsterdam for the main block but these get sold and traded so hard to say. Seems I’ve seen a fair number from the Netherlands though considering how small a country it is.

But, depending on the server’s purpose, you may see no impact on your services other than less spam. :wink:

1 Like