FirewallD Webmin page is blank, yet it's running

ID10T · March 9, 2023, 10:27pm

nftables. I’m looking at how do display what’s what now. I was barely literate in iptables. But I’m guessing this means fail2ban is working with firewalld. But, I’m still working on finding out how to verify this directly from the command line. Long time and one iteration behind. I just found where to dump the tables since firewalld didn’t even list the public table in its’ output command.

MrPete · March 9, 2023, 10:36pm

I’ve got my machine at least kinda-sorta stable now. But in reality it is NOT happy. A reboot would kill it

With the patches to fail2ban, and properly converting debian/firewalld to use iptables, fail2ban is working. The required patches:
- set FirewallBackend=iptables in /etc/firewalld/firewalld.conf
- run these two commands, and choose the iptables version from the list:
```
 update-alternatives --config iptables
 update-alternatives --config ip6tables
```
- after starting firewalld (systemctl start firewalld), load the initial default iptables chains:
```
iptables-apply /etc/iptables.up.rules
```
Also Good News – the fail2ban incremental blocking is working very nicely! My configuration (added to jail.local) is

bantime.increment = true
bantime.factor = 1

#use 22 - six weeks.
bantime.formula = ban.Time * (1<<(ban.Count if ban.Count<22 else 22)) * banFactor

# new feature: search all jails for this IP. They can't sneak in another way :)
bantime.overalljails = true

Note: I tried using @Ilia suggested virtualmin-config-system – that didn’t fix it

Here’s What Is Not Working

Although firewalld is running, the GUI isn’t working (see above for a screen shot)
Restarting firewalld causes the enhanced iptables chains to be lost. It reverts to the usual INPUT OUTPUT etc chains.

I’ll let some email flow, then try the suggestion of reinstalling firewalld from scratch.

MrPete · March 10, 2023, 12:53am

More progress! Looked at source code (/usr/share/webmin/firewalld/firewalld-lib.pl) and found that the GUI examines the exit code from a status command. For a normal user, that’s:

firewall-cmd --state
echo $?

Result: while firewalld is running according to systemctl, the state is “failed”, code 251 – “running but failed”. Now to discover what that means.

MrPete · March 10, 2023, 1:57am

SOLVED (I think). Below: how I diagnosed; issues found; potential bugs and room for improvement.

DIAGNOSING

As noted above, Webmin uses the equivalent of firewall-cmd --state; echo $? to determine whether firewalld is running and thus configurable.
I couldn’t determine what was actually causing my issues. Best way:
- stop the service (systemctl stop firewalld)
- manually run it, with debug output directed to a file:
  /usr/sbin/firewalld --nofork --debug=10 >/tmp/dbgout (debug 10 gives a LOT of output)
  NOTE: press ctrl-C after a bit to kill the process. All I wanted was startup messages
One by one, I went through the errors found, determined the problem, fixed it, and it went further
Once all errors were resolved, firewalld started up properly!
IMPORTANT HINT: the errors don’t necessarily relate directly to the symptoms seen. Just solve the errors, and seemingly unrelated problems “resolve themselves”

ISSUES FOUND

I was getting complaints that various ports were already configured. But they weren’t! The real problem: a public port enablement of ports 1025-65535. That causes all other port enablements in that range to fail (in Debian 11) but is not an issue in Debian 10.
- Workaround: I deleted that from /etc/firewalld/zones/public.xml
There were several “direct” rules for fail2ban. Not sure where they came from, but I decided to clean everything up… and now it seems to work ok. Haven’t beaten up on reboots and such but I sense it will be ok.
- The fixes…(after stopping fail2ban)
  - #1: edit /etc/firewalld/direct.xml and remove any rules there
  - #2: edit /etc/fail2ban/jail.local and insert banaction that uses firewalld rather than iptables directly
```
banaction = firewallcmd-ipset
banaction_allports = firewallcmd-ipset
```

POTENTIAL BUGS, ROOM FOR IMPROVEMENT

If Webmin/Virtualmin is creating or allowing massive port ranges… especially overlapping ranges, stop it.
Ensure /etc/fail2ban/jail.local by default has the above banaction settings
It would help if the firewalld GUI displays any error from firewall-cmd --state including the exit code and a link to the exit code list. Seeing nothing on that page doesn’t tell me much

Hints for others:

if systemctl status firewalld shows ANY errors, go find out what is causing them. Until those are repaired, it will simply not work.
Overlapping port ranges is apparently a Big Deal in Debian 11.

Joe · March 10, 2023, 2:08am

We create that rule, but I don’t understand what’s wrong with it? What is the actual error?

vending_makina · March 10, 2023, 2:44am

I’d like to share my experience… I have 2 servers running plain Debian 11 with only Virtualmin installed via the installation script, and they’re both working fine.

Fail2ban is monitoring and banning as it should.
FirewallD is showing the active rules in the Webmin UI.

And no special configuration was done to the system in this area.

MrPete · March 10, 2023, 10:54am

In Debian 11, firewalld complains about any port overlap. So installing anything that uses port numbers above 1024 will cause a failure due to 1025-65535 already being allowed.

While I’m here (@Ilia this is for you too), I noticed in the upgrade that two presumably important packages are in the “autoremove” list: installed yet supposedly no longer needed?!

webmin_virtualmin_nginx
webmin_virtualmin_nginx_ssl

(This server’s running nginx!)

MrPete · March 10, 2023, 10:58am

What jails are you using in fail2ban?

(Interesting: your FirewallD does NOT show ports 1025-65535 allowed, yet @Joe says Virtualmin creates such a rule. I wonder how it was removed on your system…)

ID10T · March 10, 2023, 3:05pm

I double checked mine because that seems overly broad from what I remembered. I get:
49152-65535

Ilia · March 10, 2023, 7:54pm

I changed it quite some time ago to use these ports and services only:

20/tcp 2222/tcp 10000-10100/tcp 20000/tcp 49152-65535/tcp
ssh smtp smtps smtp-submission ftp pop3 pop3s imap imaps http https dns mdns dns-over-tls

While I’m here (@Ilia this is for you too), I noticed in the upgrade that two presumably important packages are in the “autoremove” list: installed yet supposedly no longer needed?!

This is a common (on distro-upgrades) apt misbehaving issue. Apparently something got broken (perhaps, certain packages got deleted ignoring dependencies using dpkg?). Either way, it is not Virtualmin issue. We cannot do anything about this! This is apt. I would check on apt metadata file, as it’s the easiest way to hack (fix) it.

banaction = firewallcmd-ipset

We’re already doing it.

banaction_allports = firewallcmd-ipset

That I wouldn’t want, if what it does is blocking access to all ports for offending IP address (instead of specific service). It may or may not be a desirable behaviour, nevertheless a user can easily lock him/her self out of the box completely. We don’t really want it. The point is to stop a simple non-distributed brute-force attack. FirewallD alongside with Fail2Ban can do quite successfully already.

MrPete · March 10, 2023, 9:29pm

This is simply the setup of how banaction_allports should do its thing, IF it is used by a jail filter.

I 100% agree that this should be a rare-if-ever action. However, it’s a standard thing in fail2ban, and if not configured this way, it will be left as using iptables directly which will break f2b.

MrPete · March 10, 2023, 9:31pm

That sounds perfect. I must have installed mine before that change.

Dumb question: Is there something that ought to auto-update such things, or do I need to scan through past Virtualmin updates for changes like this?

vending_makina · March 10, 2023, 10:13pm

The concept is if it is not broken don’t fix it.

Also writing an apt scriptlet code that will change the existing rules upon upgrading a package doesn’t sound like a great idea because admins could have changed the rules manually or they might be depending on them some how…

Ilia · March 10, 2023, 10:14pm

What exactly this option do?

Is there something that ought to auto-update such things, or do I need to scan through past Virtualmin updates for changes like this?

This is done during post-configuration (installation) time. Manually you could configure FirewallD with running the following command:

virtualmin-config-system -i Firewalld

MrPete · March 10, 2023, 10:51pm

Are you saying that it does not happen while updating a system with a new version of virtualmin?

Is virtualmin-config-system safe to use on an existing installed system?

Joe · March 11, 2023, 12:32am

virtualmin config-system (on an existing system with Virtualmin, it is reachable via a subcommand of the virtualmin command) can be run at any time, but what happens when you run it depends a lot of what you’ve done before it runs. It assumes it’s running on a fresh system, or at least a fresh install of the service it is configuring. All bets are off if it is not that. It does not make a bad configuration good, it only makes a non-configuration into “configured for Virtualmin”.

It doesn’t have much notion of state. I mean, it’s not a simple search/replace in most cases; it uses Webmin libraries to parse the config files and modify them in a sensible manner. It won’t leave a smoking pile of wreckage, but it also isn’t going to turn a configured system into a “default” system.

So, the question isn’t: Is it safe? It probably is safe.

The question is: Will it do something useful? It probably won’t, unless you removed the package and all config files and reinstalled the package and all config file, first (which may not be safe, but probably is for firewalld).

Joe · March 11, 2023, 12:35am

But, I should point out, that’s probably a lot more trouble than just fixing whatever minor issues you have with the firewall. Just change the rules to suit you. It’s not a critical thing. Firewalls barely do anything useful in a world-facing server…only the fail2ban-managed rules are all that useful, and it sounds like you’ve got that sorted.

system · March 20, 2023, 3:31pm

This topic was automatically closed 8 days after the last reply. New replies are no longer allowed.