Is it safe to install package updates? I mean, can it mess up something?
I am constantly checking Apache Error Log and first few months i had pretty much no errors. I wasn’t changing any settings… i only installed package updates and then few months later i am seeing thousands of errors every day.
I was checking apache access log and i noticed a lot of bots. I decided to add robots.txt and block all bots except google and bing. Turned out there are so called “bad bots” who ignore rules from robots.txt so i had to block via htaccess.
And you know what? today i have only one error.
I still have to find out how to block “PetalBot” as it somehow manages to get around block from htaccess.
Today i have 400+ errors. But all these errors in around a minute.
From: Jun 07 04:07:10 till Jun 07 04:08:35.
Errors are from around 10 IPs:
ISP: China Telecom
China: Recently reported forum spam source. (13)
ISP: China Unicom Liaoning
ISP: Netmagic Datacenter - India
ISP: Performive USA
Services: Suspected network sharing device
Recently reported forum spam source. (94)
[proxy_fcgi:error] (70007)The timeout specified has expired: [client IP:63861] AH01075: Error dispatching request to : (polling), referer: edomain.com/
[proxy_fcgi:error] [client IP:47866] AH01079: failed to make connection to backend: localhost, referer: edomain.com/
[proxy:error] (70007)The timeout specified has expired: AH00957: FCGI: attempt to connect to 127.0.0.1:8000 (*) failed
Can someone explain what is “referer”? and why in most errors this referer is my domain with “e” in front?
In screenshots i have same errors but there was referer with “n” in front of my domain.
I use Cloudflare for that. There are several bots that have that sort of behavior and completely ignore robots.txt.
I think you can do the same thing with whatever firewall you are using, you simply make a rule for it and that will stop it.
A referer is something that sends you somewhere else. For instance, if you’re on this site and I post a link to another site and you click on it, this site is the referer. It sent you there and that site sees it.
The referer is set by the client. Your server doesn’t control what referer the client sends (for browsers, it reports the actual referer, but crawlers can make up anything they way). No one other than the person who wrote or configured the software that’s making the request can tell you why they chose to send that referer. A referer can be spoofed trivially (which is why you should never depend on the referer for security).