Setting up CSF in Virtualmin keep getting empty logs

I appreciate CSF isn’t supported here but the following questions are actually more to do with Virtualmin and checking that it’s connecting up with CSF or where to check Virtualmin/Webmin settings to make sure they allow for the correct fucntioning of CSF.

I’ve successfully (seemingly) set up CSF within Virtualmin, including the admin module using the documentation instructions. Didn’t have any issues on installation. Virtualmin is running on a fully patched Ubuntu 18.04.5 running on AWS with a firewall rule in AWS set to block SSH port 22 to only accept connections from my IP address, no other rules in AWS firewall. All seems to work well. Virtualmin is set up with one domain running a Wordpress website over SSL.

Running “Check server security” within the CSF control panel returns 38/39, only returning “Check SSH on non-standard port” but otherwise no issues or errors flagged.

Running “Test iptables” in CSF control panel also returns no issues, with all tests returning “OK” and “RESULT: csf should function on this server”.

Nothing outwardly seems misconfigured or returns an error. TESTING mode for CSF is definitely off.

The issue I’m having is that clicking “View iptables log” in CSF control panel always returns “No logs entries found”, even after months of use. Similarly when I click “View lfd statistics”, it’s currently returning “No statistical data has been collected yet”. I had previously used the “Reinstall csf” button in CSF to start from scratch and it was showing (minimal) data that basically never changed and seemed to not be updating.

One of the features I’ve set up in CSF and have had working fine on another web host with CSF was the country block feature. I’ve got it set up on this server with the Maxmind ip database and again, I can’t see any errors but have no way of verifying if it’s actually doing any country level checking or blocking.

I’m wondering if CSF is in fact working correctly in the background or more likely if it’s set up correctly but some config somewhere either in CSF or Virtualmin is not connecting up properly.

I’m not sure if CSF is accessing the correct logs within Virtualmin or if there’s something in Virtualmin that I’d need to set up to get CSF able to monitor traffic in the normal way. Within Webmin>Networking , I do have the IPv4 “Linux firewall” enabled at boot since I’m assuming this is the service that CSF hooks into. I’ve tried with it enabled and disabled at boot and same issues with no log entries (that are shown anyway) in iptables or lfd statistics. FirewallD has been disabled both in Webmin and I also ran the CSF script to disable other firewalls.

In any case, I’d appreciate any advice as to how to investigate the iptables log and the lfd statistics seemingly being empty and just generally step through what needs to be in place to connect up CSF and Virtualmin and where the problem might be. Seems like it could be a simple misconfiguration somewhere.

EDIT: Interestingly, when I right click “ConfigServer Security & Firewall” and open in new tab as suggested on other threads, it opens a blank page which clearly others don’t have. The url includes “index.cgi”. Could cgi be blocked or malfunctioning be part of the issue. i.e. is the issue with CSF a display problem with it unable to access and present data rather than a back end issue?

.Hi,

We actually support CSF pretty well on UI side. Does the interface look like this for you?

As you can see, I just installed CSF a minute ago to give it a quick test and there are logs already.

There reason why this page shows nothing is that nothing gets written to /var/lib/csf/stats/iptables_log file. You need to figure this out or ask this question to CSF Forum. I think it’s due to misconfiguration of some such.

I’m wondering if CSF is in fact working correctly in the background or more likely if it’s set up correctly but some config somewhere either in CSF or Virtualmin is not connecting up properly.

Virtualmin does nothing and if you installed CSF using install.sh (generic) then it just should work out of the box, as their install script does the job.

The url includes “index.cgi”. Could cgi be blocked or malfunctioning be part of the issue. i.e. is the issue with CSF a display problem with it unable to access and present data rather than a back end issue?

This is expected as referrer isn’t set on opening a new tab (at least in Firefox.)

Check more with the following command:

csf --help

I do have the IPv4 “Linux firewall” enabled at boot since I’m assuming this is the service that CSF hooks into.

You don’t need to do much of anything or enable anything, after installing CSF using install.sh script. On contrary if you do, things may stop working, in case you do something wrong and/or what CSF isn’t expecting. After installing CSF, you no longer need any of those modules, nor Fail2Ban, you can safely disable them.

If the error persists, use their forums for searching an answer and please follow back here with us, when you find one (answer).

@Ilia Thanks for replying and offering suggestions. My interface doesn’t look like your screenshot unfortunately. I’ve attached a screenshot of mine which is blank:

Iptables log:

Lfd Statistics:

I’ve already tried reinstalling using the CSF installer script but clearly something isn’t finding the right logs or is able to connect correctly. I’ve asked about this at the CSF forums but haven’t yet had a reply.

I’m going to try completely uninstalling CSF first and then reinstalling it to see if that helps. In the meantime if you/anyone has any other ideas about what to check, permissions maybe etc then please do let me know.

EDIT: Looks like I don’t even have an iptables_log file in /var/lib/csf/stats… weird…

CSF on my system is logging at an approximate rate of an IP per second this morning. Slow day, I guess.

Do you have fail2ban disabled? It’s unnecessary with CSF running. I don’t know for sure, but I also wouldn’t be surprised if the two applications didn’t get along very well.

Richard

@RJM_Web_Design Thanks for reply. Just checked and fail2ban isn’t enabled but good idea to check. I’m going to try a full CSF uninstall and reinstall later today and see how it goes but at the moment the blank logs are a bit of a mystery as I can’t see any obvious errors and have followed installation instructions for CSF itself and the Webmin module to the letter.

@diagonali You might want to reboot your system.

I don’t remember because it’s been a long time, but it’s possible that the Webmin CSF module will install CSF if it’s not already installed. Is that true, @Ilia ?

If so, it might also align anything needed for the logs to work. But I don’t remember whether that’s how I did it.

Richard

No, I don’t think so.

If so, it might also align anything needed for the logs to work. But I don’t remember whether that’s how I did it.

Perhaps changing profiles resulted in such breakage due to what is does to logging restrictions.

Overall, this is not Virtualmin issue. It should be asked to CSF devs.

@Ilia So I’ve rebooted a few times now and used the csf uninstall script to uninstall and then re-install csf completely. I’ve also uninstalled the csf module and reinstalled it. I’m still getting nothing in the logs so I suppose it can’t access them/use them. No idea what to do from here. I wonder if anyone with the same version of CSF (14.08) and Virtualmin has the same problem or if it’s something specific to my installation (Webmin 1.962, Virtualmin 6.14, Usermin 1.812). I also wonder if AWS somehow interferes with iptables.

@RJM_Web_Design Thanks for suggestion, I did try that but it didn’t work.

Iptables logging is not on by default in ubuntu. Maybe its is or isnt on some images. But its never been on for any of my default Ubuntu going back to like 12.04. Been using CSF for years and the iptables log is blank on all my 16.04 and 18.04. I’ve never enabled it, but I think you have to enable logging in the iptables config file. Its not a virtualmin or csf setting.

@scotwnw Interesting. I actually just created a completely fresh Virtualmin installation on AWS using Ubuntu 20.04, installed CSF and same exact blank iptables and lfd statistics logs. I’ve had a look around at how to enable iptables logging in Ubuntu but found nothing very clear on the issue in how to get it up and running in a way CSF can use. Maybe the CSF forum post will get a reply. I’ll keep looking, this clearly isn’t a Virtualmin problem. Thanks!

2 Likes

Upon further reading, it looks like it logs to syslog instead of a separate file.

“Dec 15 11:50:11 sva kernel: [2340488.879084] Firewall: TCP_IN Blocked IN=br0 OUT= …”

I have lots of these is syslog, so iptables logging is on, just not in it own file.

1 Like

@scotwnw Ah, ok. So I suppose the next question would be, is this “problem” fixable? Can we redirect where CSF is looking for the log or else get the logs written to the file CSF expects? What’s very odd though is that the CSF check after installl claims that it should run fine on the server with no errors. I’m assuming then that CSF is working correctly but isn’t able to access and display the logs? I’ve looked in /var/log/syslog and cant see any entries in there like the one you gave in yours.

Witout CSF able to read the correct logs, I’m not able to monitor the country blocking settings I’ve applied (with the CSF interface) which is the reason I’ve been trying to resolve this issue.

EDIT:

So I updated my csf.conf file and set IPTABLES_LOG = “/var/log/kern.log”. After rebooting I’m now seeing entries in the View iptables log view but with a warning about it not showing anything. Also I’m still getting “No statistical data has been collected yet” under lfd statistics but find it hard to believe it hasn’t logged anything at all so clearly something up still. Maybe CSF just doesn’t like Ubuntu and it’s not possible to get the lfd statistics showing correctly.

CSF has always worked as far as blocking bad actors, out of the box on Ubuntu. That log file and stats is just extra’s in my book to help admins. I have enabled stats though.

But The graphical stats like below is a different issue. Ubuntu is missing a graphics program or stats program. Have to install that, then it will start displaying stats. I forget what its called but I will try to find it. I believe you also have to enable stats in the csf.conf.

Also note, country blocking is very demanding. Slowed my systems to a crawl just blocking RU and CN. Note the graphs below. I Had 4 months of thousands of attempts per hour. Then I enabled subnet blocking, it took a while for it learn the bad subnets, but now I only get a few per hour. And is much less demanding than blocking millions of country IPs.

1 Like

@scotwnw Thanks for the info that helps a lot. Great tip about blocking subnets rather than using country blocking. I’ve never noticed a performance issue with country blocking despite clear warnings about it’s potential performance hit but I always enable LF_IPSET so maybe that is helping to avoid any issues. Is it a case of manually watching the logs and adding offending subnets?

As an experiment I enabled ST_ENABLE for system stats and now I can click a “View system statistics” button and it does show stats visually. I guess this indicates that the necessary graphical program in Ubuntu is there is working ok? I’ve been trying to get stats to show for lfd as in your screenshot (I had this working the same on a commerical host) but not working just yet on my Virtualmin install. I’d be very grateful If you let me know if you find out what’s needed to get it up and running.

Just to clarify though, I’m getting the message:
“No statistical data has been collected yet” when clicking the “View lfd statistics” button so I’d guess that suggests it has no data to display (or can’t access it) rather than not being able to display it.

Also, I’m wondering if the following from csf.conf (current settings and set by csf) are correct or match your settings:

HTACCESS_LOG = "/var/log/apache2/error.log"
MODSEC_LOG = "/var/log/apache2/error.log"
SSHD_LOG = "/var/log/auth.log"
SU_LOG = "/var/log/messages"
SUDO_LOG = "/var/log/secure"
FTPD_LOG = "/var/log/messages"
SMTPAUTH_LOG = "/var/log/secure"
POP3D_LOG = "/var/log/mail.log"
IMAPD_LOG = "/var/log/mail.log"
IPTABLES_LOG = "/var/log/messages"
SUHOSIN_LOG = "/var/log/messages"
BIND_LOG = "/var/log/messages"
SYSLOG_LOG = "/var/log/messages"
WEBMIN_LOG = "/var/log/auth.log"

Sorry to bother you keep asking questions but one last thing - I haven’t benn able to find an answer to if csf requires the Webmin>Networking> “Linux Firewall” to be enabled on boot or disabled? I’m assuming it provides the “back end” for csf or does csf function entirely on it’s own and I need to disable “Linux Firewall”.

Mine is disabled, but on CentOS 7. Don’t know how Ubuntu does things.

As for country blocking, I rarely use it. Even in the worst-offending countries, the percentage of users who are miscreants is tiny. I find it better to reduce the number of watched trigger incidents required to block an IP or class.

That does create problems with idiot users who keep guessing at passwords rather then just calling or using whatever other lost password functionality is provided, however. If they have static IP’s or DynDNS, I add them to csf.ignore or csf.dyndns. The rest I start billing for resets after the first few. That usually persuades them of the wisdom of writing down their passwords.

Richard

No prob, will help when I can. And I think CSF would be the default for the *mins software, where it not so complicated / non user friendly. But that’s what makes it powerful.

Sounds like you have stats working. Just needs time to run/build data. And the required package is GD::Graph perl.

And my file settings.

HTACCESS_LOG = “/var/log/apache2/error.log” <--------
MODSEC_LOG = “/var/log/apache2/error.log” <--------
SSHD_LOG = “/var/log/auth.log” <--------
SU_LOG = “/var/log/messages”
SUDO_LOG = “/var/log/secure”
FTPD_LOG = “/var/log/messages”
SMTPAUTH_LOG = “/var/log/mail.log” <--------
POP3D_LOG = “/var/log/mail.log” <--------
IMAPD_LOG = “/var/log/mail.log” <--------
IPTABLES_LOG = “/var/log/messages”
SUHOSIN_LOG = “/var/log/messages”
BIND_LOG = “/var/log/messages”
SYSLOG_LOG = “/var/log/messages”
WEBMIN_LOG = “/var/log/auth.log” <--------

Of which the important and only ones in use are with the arrows becasue /var/log/messages is not used by Ubuntu anymore. The above covers mail, http, and ssh/sftp. So if you have other stuff open and running and logged elsewhere, it should be added. For example, if you use plain ftp and it logs attempts somewhere other than /var/log/auth.log. But almost all “login” info is logged to /var/log/auth.log in Ubuntu by default. So normally not necessary to add other files.

The “liniux firewall” link there is just a front end for existing iptables rules. But DO NOT edit rules there. Edit only in CSF. If you where not using CSF and where using iptables alone, then that is where you would edit rules. And since CSF is just a front end for Iptables, the rules show in both places. You can disable that webmin module in webmin config page. Which will remove it from the menu.

@scotwnw Ok so your file settings are the same as mine and when I checked by running sudo apt install libgd-graph-perl for the perl module, it was already installed so no issues there. I think I may need to just wait over the next few days to see if I get anything showing for lfd blocking statistics.

To be super clear: I’m assuming I leave “Activate at boot” to “yes” for the “Linux firewall” under Webmin>Networking as again I’m assuming it enables the iptables feature? If “Activate at boot” is “No”, would this cause an issue with CSF? I understand that CSF takes over managing the rules so not to edit them manually myself.

Thanks

I do not have “linux firewall” listed in my bootup-shutdown webmin page. Only things firewall related are csf stuff and UFW. Of which ufw is disabled. And I did not remove or disable linux firewall. So Im not sure what program yours is referring to.

Its the one here:

I’ve got it set to “Activate at boot”, which I think means that it “enables” the iptables feature in Virtualmin which CSF uses? CSF doesn’t complain when it’s not activated at boot so I don’t know the actual answer as to what this setting does.