How to Start Fail2ban with Python3 on Centos 7?

DavidLaw · May 27, 2020, 1:53am

I have a memory issue with Fail2ban, as time passes it consumes more memory and without a reboot gets to above 2GB memory used. I’ve read it might be related to Python2, so trying to upgrade to Python3. I understand Centos 7 requires Python 2.75, so assuming I have to install Python3 as well.

Godaddy VPS: Linux 3.10.0-1127.8.2.el7.x86_64 on x86_64
CentOS Linux 7.8.2003
Python 2.7.5
Fail2ban 0.10.5

It’s a relatively new VPS and only upgraded Python and Fail2ban via the standard Virtualmin Software Package updates.

Fail2ban is working, but I think it’s got a memory leak.

Via Virtualmin Software Package Updates I’ve installed
python3 3.6.8-13.el7
fail2ban-all 0.10.5-2.el7

Despite installing fail2ban-all 0.10.5-2.el7 the version running “fail2ban-client version” is still reported as 0.10.5.

I don’t know how to change the Virtualmin setup to load the above instead of Python2 etc…

Under “Virtualmin Configuration” - “Advanced Options” I found the “Path to python command” option, is this where I add Python3 location and if so what would I add? I could only find one page in Google for the search:

Virtualmin “Path to python command”

And it doesn’t say what the format is or the implications of changing this option.

I’d guess the format is:

/usr/bin/python3

Thanks.

David

Joe · May 27, 2020, 3:47am

The -2 is a package release version (there was no version change to fail2ban, if was a patch added by the packager). The contained software is 0.10.5, as indicated.

It’s probably fine. fail2ban uses an odd memory mapped disk file for storing data which grows quite large, but the resident size is usually normal (40-200MB), and not indicative of problems.

Changing to running it under python3 is not easy or advisable.

Let’s just confirm you actually have a problem. What is the RSS/RES memory usage for fail2ban? (Look in top or htop.)

DavidLaw · May 27, 2020, 11:45am

Htop snapshot:

  1  [########################*******                        48.6%]   Tasks: 109, 73 thr; 2 running
  2  [##############************                             41.7%]   Load average: 4.34 4.19 3.33 
  Mem[||||||||||||||||||||||||||||||||||||||||||||*****2.71G/3.70G]   Uptime: 13:20:08
  Swp[                                                       0K/0K]

  PID USER      PRI  NI  VIRT   RES   SHR S CPU% MEM%   TIME+  Command                                                                       
 1319 root       20   0 2317M 32612 10568 S  0.7  0.8  1:20.63 python -s /usr/bin/fail2ban-server -xf start                  
  871 nydus      20   0 1173M 39636  3664 S  0.7  1.0  2:17.66 python3.5 /opt/nydus/pyvenv/bin/nydus-ex-api
 1534 nydus      20   0 1173M 39636  3664 S  0.7  1.0  1:05.74 python3.5 /opt/nydus/pyvenv/bin/nydus-ex-api
26338 apache     20   0  476M 16380  3892 S  0.7  0.4  0:00.06 httpd -DFOREGROUND
 1308 root       20   0 2317M 32612 10568 S  0.7  0.8  0:13.40 python -s /usr/bin/fail2ban-server -xf start
 1359 root       20   0 2317M 32612 10568 S  0.7  0.8  0:12.03 python -s /usr/bin/fail2ban-server -xf start
  878 root       20   0 2317M 32612 10568 S  0.0  0.8  7:20.30 python -s /usr/bin/fail2ban-server -xf start
 1580 mysql      20   0 1902M  278M  5840 S  0.0  7.3  6:08.25 mysqld --basedir=/usr --datadir=/var/lib/mysql --plugin-dir=/usr/lib64/mysql/p
 1586 root       20   0  412M  6936  5032 S  0.0  0.2  0:21.86 rsyslogd -n
 1352 root       20   0 2317M 32612 10568 S  0.0  0.8  0:19.88 python -s /usr/bin/fail2ban-server -xf start
 7775 **site**  20   0  454M 65744 44532 S  0.0  1.7  0:41.24 php-cgi
 1413 root       20   0 2317M 32612 10568 S  0.0  0.8  2:08.65 python -s /usr/bin/fail2ban-server -xf start
    1 root       21   1 52480  4144  2100 S  0.0  0.1  1:55.10 systemd --switched-root --system --deserialize 22
  496 dbus       20   0 58256  2236  1536 S  0.0  0.1  1:22.59 dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activat
 2198 **site**  20   0  454M 64320 43576 S  0.0  1.7  0:33.53 php-cgi           
  350 root       20   0 39480  5580  5204 S  0.0  0.1  0:55.73 systemd-journald
  504 root       20   0 26488  1696  1332 S  0.0  0.0  0:41.56 systemd-logind
  503 polkitd    20   0  598M 10140  2248 S  0.0  0.3  0:24.23 polkitd --no-debug
 1602 root       20   0  412M  6936  5032 S  0.0  0.2  0:16.68 rsyslogd -n
  534 root       22   2  354M 30424  3792 S  0.0  0.8  0:13.14 python2 -Es /usr/sbin/firewalld --nofork --nopid
  524 polkitd    20   0  598M 10140  2248 S  0.0  0.3  0:13.18 polkitd --no-debug                          
 1384 root       20   0 2317M 32612 10568 S  0.0  0.8  0:19.13 python -s /usr/bin/fail2ban-server -xf start
20131 apache     20   0  477M 18412  4688 S  0.0  0.5  0:00.28 httpd -DFOREGROUND
28509 apache     20   0  476M 16292  3720 S  0.0  0.4  0:00.02 httpd -DFOREGROUND
18663 mysql      20   0 1902M  278M  5840 S  0.0  7.3  0:00.45 mysqld --basedir=/usr --datadir=/var/lib/mysql --plugin-dir=/usr/lib64/mysql/p
 8491 mysql      20   0 1902M  278M  5840 S  0.0  7.3  0:23.95 mysqld --basedir=/usr --datadir=/var/lib/mysql --plugin-dir=/usr/lib64/mysql/p
 1401 root       20   0 2317M 32612 10568 S  0.0  0.8  0:22.21 python -s /usr/bin/fail2ban-server -xf start
13481 **site**   20   0  155M  2116   708 S  0.0  0.1  0:00.11 sshd: **site**@pts/0
 1828 mysql      20   0 1902M  278M  5840 S  0.0  7.3  0:43.94 mysqld --basedir=/usr --datadir=/var/lib/mysql --plugin-dir=/usr/lib64/mysql/p
 1302 root       20   0 2317M 32612 10568 S  0.0  0.8  0:26.76 python -s /usr/bin/fail2ban-server -xf start
 1338 root       20   0 2317M 32612 10568 S  0.0  0.8  0:18.68 python -s /usr/bin/fail2ban-server -xf start
25563 apache     20   0  477M 17352  3988 S  0.0  0.4  0:00.08 httpd -DFOREGROUND                                                            
  417 root       16  -4 55532  1016   568 S  0.0  0.0  0:00.17 auditd
 1368 root       20   0 2317M 32612 10568 S  0.0  0.8  0:19.58 python -s /usr/bin/fail2ban-server -xf start
  416 root       16  -4 55532  1016   568 S  0.0  0.0  0:03.95 auditd

I changed usernames to site for security.

The VPS has 4GB RAM.
Real memory 1.89 GiB used / 1.64 GiB cached / 3.7 GiB total

From Virtualmin Running Processes
Fail2ban is using 2.2GB RAM.
CPU load averages: 4.41 (1 mins) , 4.22 (5 mins) , 3.30 (15 mins)
CPU type: Intel Core Processor (Haswell, no TSX, IBRS) (2394 MHz) , 2 cores

It’s a relatively new VPS with roughly 60 domains which I’m still trying to optimise for performance, optimising a server isn’t my area of expertise.

I’m new to Fail2ban and have recently set it to scan all access_log files for wp-login.php and xmlrpc.php access to reduce wasted resources.

The Fail2ban WordPress filter I built is scanning 40 access_log files which isn’t the best setup performance wise. Currently looking for a way to log wp-login.php and xmlrpc.php access to a custom log file via a plugin, but not got it working as I want it yet so keeping this setup until I get the custom log working.

Although I’m concerned at the memory usage of Fail2ban I’m more concerned at CPU load averages being relatively high, it tends to run at around 2 which I’m not happy about. Despite this the websites on the VPS are running fast, since adding the new WordPress Fail2ban rules it’s got faster, so blocking IPs which are trying to access wp-login.php and xmlrpc.php has made a positive difference to overall server performance.

I regularly test my sites with tools like Google Lighthouse and they load in under 2 seconds, so the VPS is doing what I want it to do. So I’m looking for possible problems to fix and the Fail2ban high RAM usage seemed like a problem, if it isn’t I’ll go looking for another problem

Thanks for your insights.

Joe · May 27, 2020, 12:04pm

fail2ban is using 33MB. You’re fine.

system · May 31, 2020, 12:04pm

This topic was automatically closed 4 days after the last reply. New replies are no longer allowed.