Regex experts enter here

General Discussion
1

Ok thanks for your time. I have been mucking around with this for about a day and have come to the conclusion I must be stupid.

Issue at hand
We do from time to time have some people with to much time on their hand hammer away at xmlrpc.php (wordpress function). Example of logs below

172.173.116.234 - - [17/Aug/2023:16:38:20 +1000] “POST //xmlrpc.php HTTP/1.1” 200 5942 “-” “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/95.0.4638.69 Safari/537.36”
172.173.116.234 - - [17/Aug/2023:16:38:21 +1000] “POST //xmlrpc.php HTTP/1.1” 200 5942 “-” “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/95.0.4638.69 Safari/537.36”
172.173.116.234 - - [17/Aug/2023:16:38:23 +1000] “POST //xmlrpc.php HTTP/1.1” 200 5942 “-” “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/95.0.4638.69 Safari/537.36”
172.173.116.234 - - [17/Aug/2023:16:38:25 +1000] “POST //xmlrpc.php HTTP/1.1” 200 5942 “-” “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/95.0.4638.69 Safari/537.36”
172.173.116.234 - - [17/Aug/2023:16:38:27 +1000] “POST //xmlrpc.php HTTP/1.1” 200 5942 “-” “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/95.0.4638.69 Safari/537.36”
172.173.116.234 - - [17/Aug/2023:16:38:28 +1000] “POST //xmlrpc.php HTTP/1.1” 200 5942 “-” “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/95.0.4638.69 Safari/537.36”
172.173.116.234 - - [17/Aug/2023:16:38:30 +1000] “POST //xmlrpc.php HTTP/1.1” 200 5942 “-” “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/95.0.4638.69 Safari/537.36”
172.173.116.234 - - [17/Aug/2023:16:38:32 +1000] “POST //xmlrpc.php HTTP/1.1” 200 5942 “-” “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/95.0.4638.69 Safari/537.36”
172.173.116.234 - - [17/Aug/2023:16:38:34 +1000] “POST //xmlrpc.php HTTP/1.1” 200 5942 “-” “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/95.0.4638.69 Safari/537.36”
172.173.116.234 - - [17/Aug/2023:16:38:35 +1000] “POST //xmlrpc.php HTTP/1.1” 200 5942 “-” “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/95.0.4638.69 Safari/537.36”
172.173.116.234 - - [17/Aug/2023:16:38:37 +1000] “POST //xmlrpc.php HTTP/1.1” 200 5942 “-” “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/95.0.4638.69 Safari/537.36”
172.173.116.234 - - [17/Aug/2023:16:38:39 +1000] “POST //xmlrpc.php HTTP/1.1” 200 5942 “-” “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/95.0.4638.69 Safari/537.36”
172.173.116.234 - - [17/Aug/2023:16:38:41 +1000] “POST //xmlrpc.php HTTP/1.1” 200 5942 “-” “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/95.0.4638.69 Safari/537.36”
172.173.116.234 - - [17/Aug/2023:16:38:43 +1000] “POST //xmlrpc.php HTTP/1.1” 200 5942 “-” “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/95.0.4638.69 Safari/537.36”
172.173.116.234 - - [17/Aug/2023:16:38:44 +1000] “POST //xmlrpc.php HTTP/1.1” 200 5942 “-” “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/95.0.4638.69 Safari/537.36”
172.173.116.234 - - [17/Aug/2023:16:38:46 +1000] “POST //xmlrpc.php HTTP/1.1” 200 5942 “-” “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/95.0.4638.69 Safari/537.36”
172.173.116.234 - - [17/Aug/2023:16:38:48 +1000] “POST //xmlrpc.php HTTP/1.1” 200 5942 “-” “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/95.0.4638.69 Safari/537.36”
172.173.116.234 - - [17/Aug/2023:16:38:50 +1000] “POST //xmlrpc.php HTTP/1.1” 200 5942 “-” “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/95.0.4638.69 Safari/537.36”
172.173.116.234 - - [17/Aug/2023:16:38:51 +1000] “POST //xmlrpc.php HTTP/1.1” 200 5942 “-” “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/95.0.4638.69 Safari/537.36”
172.173.116.234 - - [17/Aug/2023:16:38:53 +1000] “POST //xmlrpc.php HTTP/1.1” 200 5942 “-” “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/95.0.4638.69 Safari/537.36”

So I thought about banning them with fail2ban, sounded simple. I have tried a few examples I found on the interwebs but none seem to be working.

My regex currently looks like this

[Definition]
failregex = ^ .*POST .xmlrpc.php.
ignoreregex =

Obviously I’m missing something because its not matching. So if someone out there knows what my issue is I’m all ears.

Thanks
Michael

2

I am also struggling with fail2ban on my Virtualmin server.

Filters and jail settings from earlier Centos servers simply don’t trigger even when set to detect only 1 occurrence. Same with supplied jails.

The only one that does trigger at present is for Dovecot which is mildly interesting as I moved dovecot to it’s own log file.

I haven’t made time to investigate more thoroughly - was hoping I wouldn’t have to,

This is what I used on an old server though.

failregex = ^<HOST> .*POST .*xmlrpc\.php.*
3

Hi and thanks

Just gave yours a go and fail2ban wont start up after changing. The error is

[‘start’, ‘wordpress’]]] has failed. Received RegexException(“No failure-id group in ‘^ .*POST .xmlrpc.php.’”)

Some type of syntax error unfortunately.

Michael

4

This tutorial will help you to understand how Fail2Ban works

https://www.the-art-of-web.com/system/fail2ban-howto/

Use fail2ban-regex command to test your logs after any change.

Test your REGEX here https://regex101.com/

For your case I will use
<HOST> -.*\"POST //xmlrpc.php HTTP/1\.1\" .

If it is one IP or a range of IP’s you should block using firewall (iptables, nftables, ufw, firewalld) or ipset.

5

I just tried it. /etc/fail2ban/filter.d/xmlrpc.local

[Definition]
failregex = ^ .*POST .xmlrpc.php. (formatting is being screwed by Discourse)
ignoreregex =

then in jail.local

[xmlrpc]
enabled = true
filter = xmlrpc
logpath = /var/log/virtualmin/*access_log
bantime = 300
maxretry = 1

No error but it’s not detecting anything either.

I also tried

logpath = %(apache_access_log)s

6

Any chance that it is a owner/group or permission error with /var/log/virtualmin ?

7

Ill take a look, its more then 1 ip, usually between 5 and 20.

Its not causing any real issue, its just pissing me off :slight_smile: :

Many thanks
Michael

8

Ill double check

Thanks heaps
Michael

9

I gave you some clues above.

If you use the fail2ban command with your webserver access log what is the result?

10

So why the \delimiter in front of the dot in the HTTP number, but not the dot in xmlrpc.php ?

11

Sorry, I don’t understand?

12

Did you read the information from that link I gave you?

13

Did you mean the fail2ban-regex command?

Which on a single log file gave this:

Lines: 16186 lines, 0 ignored, 1838 matched, 14348 missed

So it’s capable of working by the looks.

14

Use this one. I missed the extension for the accessed file

<HOST> -.*\"POST //xmlrpc\.php HTTP/1\.1\" .

Don’t forget to put \ in front of the extension .php.

Test it here https://regex101.com/ using Python.

Regular Expression
"172\.173\.116\.234.*\"POST //xmlrpc\.php HTTP/1\.1"g

Test String
172.173.116.234 - - [17/Aug/2023:16:38:20 +1000] “POST //xmlrpc.php HTTP/1.1”

On the right side of the website you have an explanation. You will understand why I used \ in front of the dot and many more.

15

Thanks, I had the delimiter on mine and asked why you didn’t.

What I don’t get is that fail2ban-regex got 1838 matches in a single log file but fail2ban itself isn’t finding anything.

Even using the supplied apache-badbots set to 1 hit isn’t triggering.

16

Ok, thought Id test this first.

I ran

fail2ban-regex /var/log/virtualmin/healthforlife.com.au_access_log /etc/fail2ban/filter.d/xmlrpc.conf

and got this

Running tests

Use failregex filter file : xmlrpc, basedir: /etc/fail2ban
Use log file : /var/log/virtualmin/healthforlife.com.au_access_log
Use encoding : UTF-8

Results

Failregex: 5680 total
|- #) [# of hits] regular expression
| 1) [5680] -.*"POST //xmlrpc.php HTTP/1.1" .
`-

Ignoreregex: 0 total

Date template hits:
|- [# of hits] date format
| [55908] Day(?P<_sep>[-/])MON(?P=_sep)ExYear[ :]?24hour:Minute:Second(?:.Microseconds)?(?: Zone offset)?
`-

Lines: 55908 lines, 0 ignored, 5680 matched, 50228 missed
[processed in 28.07 sec]

Missed line(s): too many to print. Use --print-all-missed to print all 50228 lines

So it looks like the regex is good. (thanks)

Now for some reason f2b is not banning them … Ill have another look

Thanks
Michael

17

I have a similar behaviour, but all the other jails are working perfectly … just the new one I created ???

18

Right now it looks to be an issue with the f2b setup, nothing triggers on the /virtualmin/ log files.

19

Are any of the others filtering the /virtualmin access logs though?

20

No, all the other logs locations are written as %(postfix_log)s %(proftpd_log)s etc