fail2ban does not start on boot anymore

Virtualmin
1

Hi everyone,

(Debian 9 installation)

I think I might have spotted a bug since one of the latest updates.

fail2ban seems to be properly installed and configured but does not start anymore when the server (re)boots.

Manual starting works fine.

Webmin’s “Bootup and Shutdown” does not help: fail2ban is marked as “Start at boot” … but still does’nt.

Webmin’s “Fail2Ban Intrusion Detector” does not help either: changing the “Start at boot” leaves the option on “no”.

All the best,
Nico

2

Has it ever worked? There was a period of time where installations wouldn’t be able to start/restart fail2ban on some distros (inclulding Debian 9). That’s been fixed in a virtualmin-config update 6.0.18.

As long as you haven’t altered your fail2ban config (specifically jail.local), you can make sure you have the latest virtualmin-config package and then run:

# virtualmin config-system --include Fail2banFirewalld # systemctl restart fail2ban

This will fix a bug in the upstream packages which have a dependency on an iptables systemd unit, which doesn’t even exist on Debian/Ubuntu.

3

Hello Joe, thank you for your answer.

Yes it did work, not so long ago. Here comes a little bit of context :

  • I installed a server a couple of weeks ago. I noticed the recent “ProFTP LoadModule mod_tls.c” bug but fail2ban worked ran like a charm and started without problem after reboot.

  • I installed another server yesterday. I noticed that the ProFTP bug has meanwhile been solved (cool), but the fail2ban bug poped out.

Would it be possible that the fail2ban bug might have been reintroduced with the latest update?

Last but not least :

  • I ran the 2 commands above, unfortunately it did not help.

  • The installations I am talking about are made on freshly installed Debian 9 servers.

Thanks again,
Nico

4

Can you run:

# fail2ban-client -x start

And, tell me what happens. If it starts without error, then there’s still a systemd unit file bug. If it fails to start, it’ll give us a hint about why.

5

Sure.

While fail2ban is runing :

# fail2ban-client -x start
ERROR Server already running

While fail2ban is stoped :

# fail2ban-client -x start
2017-09-16 08:04:33,661 fail2ban.server [4102]: INFO Starting Fail2ban v0.9.6
2017-09-16 08:04:33,661 fail2ban.server [4102]: INFO Starting in daemon mode

By the way this is a test server, no problem if you need an acces to it.

–n

6

You may need to restart the firewalld systemd unit, before the fail2ban unit, as well…I don’t remember what depends on what.

7

So that response tells us it is systemd that is failing and not fail2ban. So, fail2ban config is good. Problem is with the systemd unit files. I know I fixed that in the most recent virtualmin-config.

What’s in the file /etc/systemd/system/fail2ban.service?

8

root@ns367930.ip-94-23-26.eu:~# cat /etc/systemd/system/fail2ban.service
[Unit]
Description=Fail2Ban Service
Documentation=man:fail2ban(1)
After=network.target firewalld.service
PartOf=iptables.service firewalld.service

[Service]
Type=forking
ExecStart=/usr/bin/fail2ban-client -x start
ExecStop=/usr/bin/fail2ban-client stop
ExecReload=/usr/bin/fail2ban-client reload
PIDFile=/var/run/fail2ban/fail2ban.pid
Restart=always

[Install]
WantedBy=multi-user.target

And moreover :

root@ns367930.ip-94-23-26.eu:~# systemctl status firewalld

  • firewalld.service - firewalld - dynamic firewall daemon
    Loaded: loaded (/lib/systemd/system/firewalld.service; enabled; vendor preset: enabled)
    Active: active (running) since Sat 2017-09-16 07:49:40 UTC; 24min ago
    Docs: man:firewalld(1)
    Main PID: 539 (firewalld)
    Tasks: 2 (limit: 4915)
    CGroup: /system.slice/firewalld.service
    `-539 /usr/bin/python3 -Es /usr/sbin/firewalld --nofork --nopid

sept. 16 07:49:41 ns367930.ip-94-23-26.eu firewalld[539]: WARNING: Port ‘submission/tcp’ already set, ignoring.
sept. 16 07:49:41 ns367930.ip-94-23-26.eu firewalld[539]: WARNING: Port ‘domain/tcp’ already set, ignoring.
sept. 16 07:49:41 ns367930.ip-94-23-26.eu firewalld[539]: WARNING: Port ‘ftp-data/tcp’ already set, ignoring.
sept. 16 07:49:41 ns367930.ip-94-23-26.eu firewalld[539]: WARNING: Port ‘domain/udp’ already set, ignoring.
sept. 16 07:49:41 ns367930.ip-94-23-26.eu firewalld[539]: WARNING: INVALID_HELPER: ‘nf_conntrack_ftp’ not available in kernel
sept. 16 07:52:07 ns367930.ip-94-23-26.eu firewalld[539]: WARNING: ‘/sbin/iptables-restore -n’ failed:
sept. 16 07:52:07 ns367930.ip-94-23-26.eu firewalld[539]: ERROR: COMMAND_FAILED
sept. 16 08:04:19 ns367930.ip-94-23-26.eu firewalld[539]: ERROR: NOT_ENABLED: rule ‘(’-p’, ‘tcp’, ‘-m’, ‘multiport’, ‘–dports’, ‘smtp,465,submission,imap3,imaps,pop3,pop3s’, ‘-m’, ‘set’, '–ma
sept. 16 08:04:36 ns367930.ip-94-23-26.eu firewalld[539]: WARNING: ‘/sbin/iptables-restore -n’ failed:
sept. 16 08:04:36 ns367930.ip-94-23-26.eu firewalld[539]: ERROR: COMMAND_FAILED

root@ns367930.ip-94-23-26.eu:~# systemctl restart firewalld

root@ns367930.ip-94-23-26.eu:~# systemctl status firewalld

  • firewalld.service - firewalld - dynamic firewall daemon
    Loaded: loaded (/lib/systemd/system/firewalld.service; enabled; vendor preset: enabled)
    Active: active (running) since Sat 2017-09-16 08:15:01 UTC; 3s ago
    Docs: man:firewalld(1)
    Main PID: 5686 (firewalld)
    Tasks: 2 (limit: 4915)
    CGroup: /system.slice/firewalld.service
    `-5686 /usr/bin/python3 -Es /usr/sbin/firewalld --nofork --nopid

sept. 16 08:15:01 ns367930.ip-94-23-26.eu systemd[1]: Starting firewalld - dynamic firewall daemon…
sept. 16 08:15:01 ns367930.ip-94-23-26.eu systemd[1]: Started firewalld - dynamic firewall daemon.
sept. 16 08:15:01 ns367930.ip-94-23-26.eu firewalld[5686]: WARNING: Port ‘submission/tcp’ already set, ignoring.
sept. 16 08:15:01 ns367930.ip-94-23-26.eu firewalld[5686]: WARNING: Port ‘domain/tcp’ already set, ignoring.
sept. 16 08:15:01 ns367930.ip-94-23-26.eu firewalld[5686]: WARNING: Port ‘ftp-data/tcp’ already set, ignoring.
sept. 16 08:15:01 ns367930.ip-94-23-26.eu firewalld[5686]: WARNING: Port ‘domain/udp’ already set, ignoring.
sept. 16 08:15:01 ns367930.ip-94-23-26.eu firewalld[5686]: WARNING: INVALID_HELPER: ‘nf_conntrack_ftp’ not available in kernel

9

(trying to answer here but my comment has been queued for review)

10

(trying a shorter answer)

# cat /etc/systemd/system/fail2ban.service
[Unit]
Description=Fail2Ban Service
Documentation=man:fail2ban(1)
After=network.target firewalld.service
PartOf=iptables.service firewalld.service

[Service]
Type=forking
ExecStart=/usr/bin/fail2ban-client -x start
ExecStop=/usr/bin/fail2ban-client stop
ExecReload=/usr/bin/fail2ban-client reload
PIDFile=/var/run/fail2ban/fail2ban.pid
Restart=always

[Install]
WantedBy=multi-user.target

11

That looks like it did work. Fail2ban is running! (And the errors you see in the status are merely warnings about trying to add rules that already exist, which is harmless.)

So, what’s the problem we’re trying to tackle now?

12

Hi Joe,

Unfortunatly the problem remains and and the symptoms are the same :

  • Yes fail2ban starts, but only when started manually (command line, Webmin) but it does not start automaticaly when the server (re)boots.

  • Webmin’s “Bootup and Shutdown” module does not help: fail2ban is marked as “Start at boot” … but still does’nt start at boot.

  • Webmin’s “Fail2Ban Intrusion Detector” module does not help either: changing the “Start at boot” option to “yes” has no effect (it remains on “no”)

Thanks again,
Nico

13

What does systemctl status fail2ban say?

14

Just after having reboot the server, fail2ban is not started (but can be started manually). More specificaly, this happens:
root@ns367930.ip-94-23-26.eu:~# systemctl status fail2ban

  • fail2ban.service - Fail2Ban Service
    Loaded: loaded (/etc/systemd/system/fail2ban.service; disabled; vendor preset: enabled)
    Active: inactive (dead)
    Docs: man:fail2ban(1)

root@ns367930.ip-94-23-26.eu:~# systemctl start fail2ban

root@ns367930.ip-94-23-26.eu:~# systemctl status fail2ban

  • fail2ban.service - Fail2Ban Service
    Loaded: loaded (/etc/systemd/system/fail2ban.service; disabled; vendor preset: enabled)
    Active: active (running) since Tue 2017-09-19 09:31:46 UTC; 3s ago
    Docs: man:fail2ban(1)
    Process: 2348 ExecStart=/usr/bin/fail2ban-client -x start (code=exited, status=0/SUCCESS)
    Main PID: 2369 (fail2ban-server)
    Tasks: 16 (limit: 4915)
    CGroup: /system.slice/fail2ban.service
    |-2369 /usr/bin/python3 /usr/bin/fail2ban-server -s /var/run/fail2ban/fail2ban.sock -p /var/run/fail2ban/fail2ban.pid -x -b
    `-2428 /usr/bin/python3 -Es /usr/bin/firewall-cmd --direct --add-rule ipv4 filter INPUT 0 -p tcp -m multiport --dports ssh,sftp -m set --match-set fail2ban-ssh-ddos src -j RE

sept. 19 09:31:45 ns367930.ip-94-23-26.eu systemd[1]: Starting Fail2Ban Service…
sept. 19 09:31:45 ns367930.ip-94-23-26.eu fail2ban-client[2348]: 2017-09-19 09:31:45,694 fail2ban.server [2360]: INFO Starting Fail2ban v0.9.6
sept. 19 09:31:45 ns367930.ip-94-23-26.eu fail2ban-client[2348]: 2017-09-19 09:31:45,694 fail2ban.server [2360]: INFO Starting in daemon mode
sept. 19 09:31:46 ns367930.ip-94-23-26.eu systemd[1]: Started Fail2Ban Service.

15

(I tried to answer but my comment has been queued for review again, would it be possible to remove the restriction concerning the lenght of a post?)

For the record, I freshly re-installed the server under Debian 8 (I can go back to 9 anytime if needed) and this time fail2ban does not start at all, neither on boot, nor manually.

16

Problem is here:

Loaded: loaded (/etc/systemd/system/fail2ban.service; disabled; vendor preset: enabled)

It’s disabled. So, enable it:

# systemctl enable fail2ban

Not sure why it would have changed, as we enable it during installation, but it looks like everything is working as it should be now, so enabling it should sort out your trouble.

17

There is no length restriction. It is a spam filter called Mollom. We have very little control over the parameters of how it works. It has a lot of false positives (seemingly more lately). We’ll be moving to a better forum in the future, but I just don’t have the hours in the day to tackle it right now.

“For the record, I freshly re-installed the server under Debian 8 (I can go back to 9 anytime if needed) and this time fail2ban does not start at all, neither on boot, nor manually.”

Why? Debian 9 is better in every way to Debian 8. It’s why they made a new one.

Edit: Also, I spend way more time/effort on testing and supporting the latest version of the distros we support, because they have the longest future. My efforts spent on new distros are more valuable than effort spent on old ones. Help me maximize my time by using the latest supported version of your preferred distro.

18

Hi Joe,

Sorry for the delay of this answer and thank you so much for your time.

You where right: fail2ban was disabled and enabeling it easily solved the problem.

Still, it is a little odd that fail2ban was disabled on fresh installs (I even noticed it was masked on one installation) BUT that problem has not occured anymore for about 10 days. I don’t know if you have changed anything in the install script, but fail2ban has been correctly enabled on every of my latest installations. Cool, problem solved :slight_smile:

Thanks again,
Nico

PS: I totaly agree on the fact that the latest version of a distribution should be used for fresh installs … but unfortunately I am not the only one to decide and I do not always have the last word :slight_smile: