Often vps became unreachable (Fail2Ban)

I have two VPS and I experienced two similar behavior by VPS with virtualmin (debian).

Similar to this: often can't login to :10000

I think is firewall problem related, because after a reboot all return to normal. The problem occurs when there are many request to login or about scan.
For example one time I have tried twice to FTP and now I had a scan for virus from a website that test the server.

After this I cannot enter the vps, neither web or SSH, always respond with “timeout”.

Maybe there are some “ban” by Fail2Ban?

You’re probably running out of memory. It’s almost certainly not fail2ban, unless you’re entering the wrong password over and over for every service (I think the default is 5 failures, and it only blocks an IP for a few minutes to start). If it somehow is fail2ban, then the solution is to learn your password or use a password manager and copy/paste the right password and it will stop being a problem.

But, check dmesg for out of memory errors. It’ll kill user processes to avoid a kernel panic. The solution to that is get more memory or add more swap (but swap is slow), or reduce the need for memory (e.g. don’t use clamav).

Cannot be memory because the VPS is new with 8 GB of RAM and 6 GB of swap.
I am using only 2 GB of RAM.

This this was not a problem of password.
I had to do a test for malware from a website, which have done a scan of server. After that I was unable to login (web and SSH) but websites were online

check fail2ban.log for clues regarding your ip… could be fail2ban, but only logs will tell…

I did not find my ip, there are many other in the time when problem occurred

I have this into “firewalld” log’s file
2020-02-14 12:34:37 ERROR: COMMAND_FAILED: ‘/usr/sbin/iptables-restore -w -n’ failed: iptables-restore v1.8.2 (nf_tables): invalid port/service imap3' specified Error occurred at line: 2 Try iptables-restore -h’ or ‘iptables-restore --help’ for more information.

Looks like FirewallD doesn’t recognize what imap3 is. Is that a typo? Shouldnt it be imap? Or try the actual port number in the firewall instead of the “service” name.
Here default firewalld on new ubuntu install.

I have the same, mail works however.

That is not related to the “timeout” problem btw. Only occurs when there are many try to login (a few). maybe I should set my ip into whitelist?

It’s a typo. It’s supposed to have been fixed months ago, but I missed pushing it into a package. New virtualmin-config package goes out soon, but you’ll just want to change it to imap.

thanks, but that for now is not a problem. mail seem to work

for the fail2ban I did find my ip into logs, maybe firewalld blocks all ip when there are too much attemps?

Fail2ban blocks the specific IP that has login failures, never all IPs (you could make rules to have it block all IPs, but that’s not what is happening in a default Virtualmin system).

what could cause that behavior? (if not fail2ban)

Behavior:

  • All virtual host works and are accessible
  • SSH/Webmin give “error timeout” and is impossible to login
  • After reboot all return to normal

I had similar problem on Ubuntu 16… I found out I will need to upgrade to new Ubuntu with newer version of fail2ban… for now I just uninstalled fail2ban…

I have Debian 10 with newer fail2ban, I dont want to uninstall because I think is useful to preserve memory (I guess)

@wolfseo just out of the interest, that website which testing your site - how does it works? Perhaps I am wrong but when you run that test it can hog up your resources which are used to test your site, be it connection via ssh or something. Does the test ever finished?

I would bet on it is not fail2ban - that would not ban your ip, it would ban the site which is performing test on our server but not you.

btw, if you deployed site your self what you can do to check if there is anything changed, you can create md5 hash checklist.txt file for everything in your public_html and when you want to see if anything was changed ever you just compare checklist.txt against md5 check. This way you would be able to see if anything was changed in php-s or any other files and folders withing your /public_html and results will tell you, if the hash is miss-match means if you did not edit those files, someone or something did. If you run worpdress, joomla or any other cms this is very useful but also very fast and light way to check if someone changed anything inside of your files… keep in mind that cms mostly keep content in databases however uploaded images etc will be shown like new files means not edited.

One time occurred with FileZilla, I put the wrong password, I have received a message “timeout” and then I could not access into webmin from browser and SSH from putty

@wolfseo ah I see… I guess you had been ban by fail2ban for 5 minutes if you did not changed rules from default - I guess you need to a bit relax fail2ban rules… I mean tell fai2ban in your regex to ban after at least 3 fails… however I know for the fact filezilla will try to reconnect couple of times which even means, that if you use default fail2ban regex rules, you might get banned which is expected to happen. Next time just wait 5 to 10 minutes before you would connect anything from your network…

Yes maybe that time I have re-try with filezilla clicking the login button few times.
Is there a config file to change default values? (failt2ban)

edit.
I checked
Maybe is that: I have 48h for Email and 10m for max retry.
Next time I’ll try to wait

@wolfseo I normally use terminal to do all of this but virtualmin have nice gui for this job too…

to edit jail timing for each jail and number of fails before IP gets banned you need to edit filter action jails under webmin -> networking -> fail2ban… tab (click on image to enlarge)

click on jail name (click on image to enlarge)

enter values that you like and hit save button. You can also enter your own public ip to be never blocked. After that just to make sure rules are applied right a way, you might restart fail2ban with orange button as shown on above screenshot. Normally my ban times are for weeks or months depend on what chicken is trying to… (click on image to enlarge)

give it couple of hours or perhaps 24 hours and check how it is doing under webmin -> networking -> linux firewall tab. Here you can click on reject word for desired IP and delete or edit the rule… with immediate effect (click on image to enlarge)

I hope so this would help to you somehow.

I have less rules (but there are sshd and proFTP that caused my ban)


should I add your rules to yes? (apache?) could be good to performance?