IPtables & Fail2Ban - Apply Configuration results in duplicates

[Rory_Bremner1]

🛈 SYSTEM INFORMATION

Operating system Ubuntu Linux 20.04.3
Webmin version 1.984 Virtualmin version 6.17-3 Authentic theme version 19.84.7
Kernel and CPU Linux 5.4.0-96-generic on x86_64
Fail2ban 0.11.1
|------------------------------|-------------------------------|
Hi all,
Just reporting that in the latest version of Webmin, I have noticed a behavior that I never seen before.
Up until August of 2019 I am positive that this was not happening. I did extensive tests back then on exactly this functionality and was working well (or maybe I had too few entries that I did not see it). Since last week when I upgraded to Ubuntu 20.04 and the latest Webmin, I am seeing this problem.

Essentially, when fail2ban is running it creates iptable rules and chains.
When I go to Webmin > Networking > Linux Firewall I click <Revert configuration> to show me all the iptables current rules. This works just fine.

The problems start as soon as you click <Apply Configuration>. You may not notice it because it is not apparently obvious - you have to run **/usr/sbin/iptables -S** to realize what is going on or click <Revert Configuration> and you will see the big mess!

It duplicates ALL fail2ban chain rules and INPUT actions. You get doubling each time you click <Apply>.

How I painfully deal with this

systemctl stop fail2ban

# Clear Fail2Ban Chains
/usr/sbin/iptables -S | grep -P '\-A f2b-.*' | cut -d ' ' -f 2 | sort -u | awk '{print /usr/sbin/iptables -F $1;}' | /bin/sh

# Clear Fail2Ban Rules
/usr/sbin/iptables -S | grep -P '\-A INPUT.*f2b-.*' | sed 's/^-A //' | awk '{print "/usr/sbin/iptables -D "$0";"}' | /bin/sh

systemctl start fail2ban
# Now click <Revert Configuration> and fail2ban will bring your rules back.
# If you have many rules you may need to click it a second time to see all of them.

Conclusion
Fail2Ban seems to be actively managing its iptables entries.

Should Webmin be disregarding F2B entries when we click <Apply Configuration>?
Could I implement my commands to the actions of the <Apply Configuration> button? How?
What are your thoughts?

[unborn]

I think - you need to update your regex configurations… last time I moved from debian 8 to debian 10 I had every ban and chain duplicated in iptables and ram was way to high - just rewrite my regex in files and its all fine. - also check your f2b logs for this…

[Rory_Bremner1]

UPDATE:
I noticed soon after posting, that all my previous rules were all gone. Only F2B had rules in my firewall - everything else was gone with the upgrade to Ubuntu 20.04 I guess.
So what I did was to restore my previous rules from a backup (/etc/webmin/firewall/iptables.save) and restart F2B which brought back its own.

Since my previous rules came from firewalld, as soon as I restored them, Webmin started warning me with a notice at the top of the 'Linux IPTables Firewall' page about the fact.
Just above that, there was another notice telling me 'External managed rules detected. Activate "Directly edit firewall rules" or your firewall rules may break.'

Clicking the link takes me to the firewall config (same thing as manually clicking the cog button at the top left of the same page). It is not apparent was I should do there. What I ended up doing was to choose from the drop down list 'Configuration category:' > 'IPv4 configuration' and choose 'YES' for **Directly edit firewall IPv4 rules instead of save file?**.

This stopped the previous message appearing and also at the bottom of the page I could now see only one button <Save Configuration> Click this button to save the configuration above into the permanent firewall configuration file (in the place where before I had 2 buttons <Apply Configuration> and <Revert Configuration>).

Good news is that now,
my rules from F2B do not duplicate themselves anymore no matter what I try.

Strange, but OK news is that now,
the iptables.save file still gets saved when I click <Save configuration> even though I chose Directly edit ...instead of save file.

So all is well again.