All of My Blocklists are now Free -- at Least for the Time Being

General Discussion
1

The sheer volume of attacks in the past month or so have persuaded me that the ethical thing to do is to make all my blocklists free, in the interest of Interwebs safety. I’ll take donations from visitors to the site if people want to contribute, but I’m not charging for the lists right now.

If things calm down, maybe I’ll revert to the freemium model. But for now… Well, it’s a jungle out there.

Here are the links to the formerly subscription-only, paid lists:

https://www.rjmblocklist.com/sizzling/freships.txt

https://www.rjmblocklist.com/sizzling/worst.txt

Richard

4 Likes
2

What are these blocklists for? Email, web attacks, whatever?

3

The lists began as as a deduplicated database of IP addresses blocked by CSF on a cohort of servers that I own or manage, to be shared between the servers in the cohort. So the bulk of the IP addresses are guilty of things that CSF detects, such as:

  • Connection attempts on ports that have been changed to non-standard ones or are unused on the servers (SSH, RDP, MS SQL, etc.).
  • Repeated login failures (POP3, IMAP, cPanel, Webmin, etc.).
  • Distributed attacks of any kind.
  • Port scans.

I’ve also added detection of common Web-based mischief, including:

  • Attempts to access non-existent CMS login pages (for example, attempts to access WordPress login pages on non-WordPress sites).
  • Attempts to exploit current CMS vulnerabilities.
  • Web-based SQL injection attempts on actual forms or honeypots.
  • Web spam and form spam submissions on actual forms or honeypots.

Occasionally I manually add the IP addresses of especially annoying email spammers, but that’s the exception rather than the rule. I usually report IP addresses of email spammers and scammers to SpamCop.

IP’s guilty of multiple kinds of mischief are deduplicated and only appear once in each list, with retention of three to four days from the most-recent mischief. If they cease misbehaving they automatically fall off the list and are rehabilitated.

The IP addresses in the worst.txt list are guilty of repeat offenses and tend to stay on the list for a while because they commit new offenses before they have a chance to fall off the list. One might consider them the incorrigible ones who defy rehabilitation.

All of these IP addresses are also reported to AbuseIPDB and appear on my AbuseIPDB page with a bit more information than appears on my blocklists, which are just lists of IP addresses with no further explanatory information. The information on AbuseIPDB is also generic in nature, but does include the broad category of mischief each IP was guilty of.

My AbuseIPDB page also includes manually-reported form spam that made it past my filters, as well as occasional email spam that made it past SpamAssassin and hadn’t already been reported to AbuseIPDB.

Richard

1 Like
4

lol I block millions of IP’s already. This would not even touch what I block and I still see many many failed attempts at gaining access. I have over 1000 ip ranges that I block and keep adding more and more. I see people doing nefarious things all the time. Many of these users use virtual machines hosted in the cloud on different server farms located all over the world. Believe it or not you see more and more attempts at hacking daily from new sources.

1 Like
5

I’m looking forward to the one-million mark. Unfortunately, I only made my lists public about a year ago. It was one of those things I thought about for about five years before getting around to actually doing it. If I’d done it when I first thought about it and registered my AbuseIPDB account, I’d be somewhere around 1.5 million by now.

6

I hear that hrmmm if your interested I can share my list with you not a problem… Just let me know it does contain ranges so dunno how that all figures in for you but for me it works. I am not a second chance kinda guy I don’t dig people using cloud hosted virtual pc’s to access my site anyways… But your mileage may vary. And if your out of country well… chances are I won’t be doing business with you so lets say an ISP switches out IP’s from a country other than North America I leave it on the list. If you think it’d be helpful I’d share with you if not well… No point.

1 Like
7

Thanks, but our philosophies are different. My emphasis is on rapid rehabilitation.

I’ve inherited IP addresses that were still on blocklists because of what someone did years prior. Depending on who manages the list, it may be almost impossible to get de-listed.

For example, Microsoft and Verizon are scam artists. They both want you to buy into paid monitoring services before they delist your IP. It’s one of their rackets.

For Verizon, getting someone at AT&T to de-list the IP works. They share the same list, but AT&T doesn’t try to scam you. Asking how to go about it on their customer support forum usually gets a reply by one of their techs within a day or two, but being a customer and complaining that you can’t receive mail or SMS from an address or domain on the blocklisted server is faster.

For Microsoft, I just block all of their IP ranges and inform them by sending them the same email that they sent me, with the roles reversed. That gets the case escalated and the IP de-listed within an hour or two, on average. You can get an approximate list by scoping out the SPF records for all of their domains (live, hotmail, outlook, etc.)

I have to give Microsoft credit for balls, if nothing else. The combined Microsoft email services are second only to Gmail as sources of incoming spam. (Bluehost is a close third.) Yet Microsoft, the second-biggest source of spam in the world, tries to extort you to get off their list. Corrupt bastards.

That’s why I focus on rapid rehabilitation. The absolute longest time an IP can stay on my lists is 96 hours from the last bad behavior. I think that’s a better approach in view of the fact that the person responsible for the mischief may not be (and usually isn’t) the owner of the IP.

Thanks anyway, though.

8

great job on that !
if i get time I may take up your offer, these kind of things the subscription model is good as you out in a lot of effort. nothing is actually free as your effort testifies.

A few years ago I started doing my own list as your common Web-based mischief, I think I have about 35,000 IP’s no doubt way short of yours ;o))

In particular if someone tried to access wp-login I knew were a scam so they were blocked on the first attempt ;o)

thanks again. one of the things I like about the Virtualmin community and the guys behind the scenes.

1 Like
9

Thanks!

Yeah, those are easy calls because of the presumption of intentionality, especially with lesser-known CMS frameworks. Anyone trying to access the login page of a site that isn’t theirs or exploit a known vulnerability in a CMS that most people have never heard of can be presumed to be up to no good, especially if you have lfd ignore the known white-hat bots.

The login failures are a tougher call. You have to consider the nature of the accounts on the server and go from there. For example, on the server hosting my own sites, the thresholds are extremely low because I’m the only one accessing the server and my own IP is whitelisted. Those thresholds wouldn’t work for a shared server. I’d catch too many innocents who just forgot their logins.

Richard

10

that’s why you have fail2ban.
wp block user-enumaration, X login tries with user, 1 try with non-existent user, etc… with wp-fail2ban. similar approach for drupal. other CMS are not difficult to implement. and they work quite well keeping a lot of ips away from loading/brute-forcing shared environments.

thanks for the lists btw, already trying it out on a small machine.

1 Like
11

My pleasure.

I use CSF firewall and let lfd handle the login failures. I have nothing against fail2ban, mind you. I’ve just been using CSF as a firewall since before fail2ban existed.

When I wrote the original blocklist scripts, the purpose was to share the database will multiple servers all running CSF. The site was built as an afterthought around something that already existed.

Richard

12

using fail2ban alongside csf … :slight_smile:
csf is actually blocking ips… fail2ban for (cms) intrusion detection… not sure how lfd catches these cms brute-force attempts… (?)

cheers,

13

It doesn’t. Those are handled by entries in .htaccess on hand-coded sites that redirect requests for CMS pages (and other resources currently being exploited) on sites where they don’t exist to a script that harvests the remote IP’s and adds them to the blocklist.

I hand code, so I have plenty of sites where none of those pages exist. Many years ago I noticed the staggering number of 404’s looking for the non-existent pages, so I redirected them to a trap. Then when I built the sharing script, I added a few lines to the trap scripts to also write the request information to the shared database.

The form spam entries are harvested from the spam blocking scripts on actual contact pages, as well as from form pages and exposed email addresses on dedicated honeypot sites.

lfd catches the brute force attacks on SASL, POP3, FTP, mail, ssh, cPanel, Webmin, RDP, etc., as well as port scans and the like.

Those are the three basic mechanisms at work, and the information harvested all winds up in the same blocklists.

I can view the database and get more specific information about the nature of a particular IP’s misbehavior, and that information is also reported to AbuseIPDB; but the public blocklists don’t differentiate between the types of attacks. They’re just text listings of all the IP’s, no matter how they wound up there.

Richard

14

that’s a bit tricky, since (eg) we host sites that have changed structure/cms/permalinks/etc over the years. so most 404s are to be expected. we never actually ban anyone based on 404…
non-existent users, brute-force logins, is what triggers bans and i consider these more accurate…
for everything else we use excellent apache-ultimate-bad-bot-blocker. it seems to refuse most of the irrelevant bots/traffic around…

don’t want to stay offtopic too long :slight_smile:
just 2c,
d.

15

Nor do I. The 404’s don’t trigger anything. The triggers are commonly-abused pages or directories that don’t exist on the site. Those would normally come up 404 on a hand-coded site, but I use .htaccess to redirect them to a trap thusly:

RewriteCond %{REQUEST_URI} /admin.php [NC,OR]
RewriteCond %{REQUEST_URI} /install.php [NC,OR]
RewriteCond %{REQUEST_URI} /lequ.php [NC,OR]
RewriteCond %{REQUEST_URI} /login.php [NC,OR]
RewriteCond %{REQUEST_URI} /setup.php [NC,OR]
RewriteCond %{REQUEST_URI} /shell.php [NC,OR]
RewriteCond %{REQUEST_URI} /user.php [NC,OR]
RewriteCond %{REQUEST_URI} /webconfig.txt.php [NC,OR]
RewriteCond %{REQUEST_URI} /wlwmanifest.xml [NC,OR]
RewriteCond %{REQUEST_URI} /wp-config.php [NC,OR]
RewriteCond %{REQUEST_URI} /wp-contacts.php [NC,OR]
RewriteCond %{REQUEST_URI} /wp-login.php [NC,OR]
RewriteCond %{REQUEST_URI} /xmlrpc [NC,OR]
RewriteCond %{REQUEST_URI} /xmlrpc.php [NC,OR]
RewriteCond %{REQUEST_URI} ^/admin/(.*)$ [NC,OR]
RewriteCond %{REQUEST_URI} ^/administrator/(.*)$ [NC,OR]
RewriteCond %{REQUEST_URI} ^/config/(.*)$ [NC,OR]
RewriteCond %{REQUEST_URI} ^/blog/(.*)$ [NC,OR]
RewriteCond %{REQUEST_URI} ^/cms/(.*)$ [NC,OR]
RewriteCond %{REQUEST_URI} ^/data/(.*)$ [NC,OR]
RewriteCond %{REQUEST_URI} ^/demo/(.*)$ [NC,OR]
RewriteCond %{REQUEST_URI} ^/fckeditor/(.*)$ [NC,OR]
RewriteCond %{REQUEST_URI} ^/.git/(.*)$ [NC,OR]
RewriteCond %{REQUEST_URI} ^/inc/(.*)$ [NC,OR]
RewriteCond %{REQUEST_URI} ^/install/(.*)$ [NC,OR]
RewriteCond %{REQUEST_URI} ^/magento/(.*)$ [NC,OR]
RewriteCond %{REQUEST_URI} ^/manager/(.*)$ [NC,OR]
RewriteCond %{REQUEST_URI} ^/media/(.*)$ [NC,OR]
RewriteCond %{REQUEST_URI} ^/news/(.*)$ [NC,OR]
RewriteCond %{REQUEST_URI} ^/old/(.*)$ [NC,OR]
RewriteCond %{REQUEST_URI} ^/plus/(.*)$ [NC,OR]
RewriteCond %{REQUEST_URI} ^/setup/(.*)$ [NC,OR]
RewriteCond %{REQUEST_URI} ^/shop/(.*)$ [NC,OR]
RewriteCond %{REQUEST_URI} ^/site/(.*)$ [NC,OR]
RewriteCond %{REQUEST_URI} ^/sito/(.*)$ [NC,OR]
RewriteCond %{REQUEST_URI} ^/staging/(.*)$ [NC,OR]
RewriteCond %{REQUEST_URI} ^/templates/(.*)$ [NC,OR]
RewriteCond %{REQUEST_URI} ^/test/(.*)$ [NC,OR]
RewriteCond %{REQUEST_URI} ^/web/(.*)$ [NC,OR]
RewriteCond %{REQUEST_URI} ^/website/(.*)$ [NC,OR]
RewriteCond %{REQUEST_URI} ^/wordpress/(.*)$ [NC,OR]
RewriteCond %{REQUEST_URI} ^/wp/(.*)$ [NC,OR]
RewriteCond %{REQUEST_URI} ^/wp1/(.*)$ [NC,OR]
RewriteCond %{REQUEST_URI} ^/wp2/(.*)$ [NC,OR]
RewriteCond %{REQUEST_URI} ^/wp-admin/(.*)$ [NC,OR]
RewriteCond %{REQUEST_URI} ^/wp-contacts/(.*)$ [NC,OR]
RewriteCond %{REQUEST_URI} ^/wp-includes/(.*)$ [NC]
RewriteRule .* /[trap-page].php?req=%{THE_REQUEST} [PT,L]

Obviously directories or pages that actually exist on a site wouldn’t be redirected. The idea is to trap people or bots that try to access common-abused pages or resources that they have no business accessing, and that don’t exist on the site.

Where the 404’s come in is that if I notice a bunch of 404’s for a particular page or resource in the stats, I research it. Usually it has to do with an unpatched vulnerability in some CMS or another; so I add it to .htaccess to trap more miscreants. The 404’s themselves don’t trigger anything except my scrutiny. :face_with_monocle:

Richard

16

Please note that as of the next regeneration (in less than an hour), the blocklist located at

https://www.rjmblocklist.com/sizzling/freships.txt

will contain a maximum of 5,000 entries, sorted by recency (most-recent first). This is an increase from the previous limit of 2,500 entries due to the increasing number of attacks being reported by the cohort of servers collecting the data.

The significance of this is that you almost certainly do NOT not want to download the whole list into your firewall, especially if you’re on a resource-constrained server. So if you’re using this list, consider setting a limit on the number of IPs downloaded.

In CSF, for example, you could set a limit of 2,500 by editing the directive in /etc/csf/csf.blocklists to look something like this:

# RJM Blocklist Fresh IP List
# Details: https://rjmblocklist.com
RJMFRESHIPS|3600|2500|https://www.rjmblocklist.com/sizzling/freships.txt

That would download the first 2,500 entries, which will be the most recent, once every hour.

To download only the most-recent 500 entries once every two hours, it would be:

# RJM Blocklist Fresh IP List
# Details: https://rjmblocklist.com
RJMFRESHIPS|7200|500|https://www.rjmblocklist.com/sizzling/freships.txt

I’m sure you get the idea.

Most servers should NOT download the whole list. That’s an awful lot of entries for the firewall to run through. But I think most admins who import third-party blocklists are savvy enough to make that decision for themselves; so I’m making up to 5,000 available and leaving it to server admins to decide how many they want to use.

The most-recent entries will always be first, no matter how many you choose to download.

Richard

1 Like
17

Due to a dramatic increase in attacks on CMS systems yesterday and overnight:

abuseipdb-120922
abuseipdb-1209221305×774 156 KB

I’ve increased the compilation frequency of the Web app attack list to hourly.

This list is limited to IP addresses that specifically attacked CMS systems and other Web-based apps by looking for vulnerable pages and directories that don’t exist on my hand-coded sites. The list is free and can be downloaded at

https://rjmblocklist.com/free/webattack.txt

Richard

1 Like
18

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.