My hosting domain and one other domain are being hit 24/7 with some bot script. This is not new but usually does not last this long and usually when i deny ip in cPanel it goes away after a day or so.
But this has been running for well over a week now all day and all night. Every day i check my error log on cPanel and every day its full of "denied by server configuration" from this one section of ips that i blocked.
This is good, but does this still use system resources, i mean can they still affect my latency by running this. They have to know by now they are being blocked.
Yes it does, in high amounts of course - if it reaches the server then the server is going to be hit.
Ahardware firewall on the network is the only thing that can stop that.
The less your server does in response, the less it will be impacted - so any "ignore" possibilities are always best, but of course an ignore is no good if the bot still does damage with minimal response.
Do you mean that you added the following line to .htaccess?
deny from 180.76.
If so, this isn't the correct way to ban the 180.76.*.* IP range because with this line you blocked all IP addresses containing 180.76. and not only IP addresses starting with 180.76.
For example 184.108.40.206 would be blocked as well.
The correct way to block the 180.76.*.* IP range from .htaccess it this line:
deny from 220.127.116.11/16
★Network Panda::Web Hosting SSD Powered::Reseller Hosting ★ Instant activation, fast servers, SSD disks, cPanel, Softaculous 1-click apps installer, daily backups, SSL certificates ★ Multiple hosting locations: USA, Canada, France, UK, Germany, Italy, Spain, Poland, Finland
It may take server resources to check the rules...but never take as much if the bots can really hit your website pages. Perhaps you can ask your host to deny Baidu network in the server firewall,because I really don't think most people like that naughty crawler.
If you set up deny IP rules through cPanel, then you are actually setting up rules in .htaccess. The stuff in .htaccess tends to be pretty efficient. There will be some overhead, but far less than letting a bot pound on PHP pages in a site.
Yes i agree we have to take a stance and draw a line in the sand at some point. I have used cPanel deny and htaccess for a long time regardless of getting ridiculed by others for doing so.
I am now building a site for this to pull some free resources and also to place some of my own experience on this. I just started it today so it is not done yet with content but it is a start. Just trying to get some stuff all in one place for people.
I will have all kind of tips and stuff that may help people and its totally free. And i will do my best to verify those tips as best i can.
The site name is
Because i have had it with these idiot spammers lol..
I was just curious if that was the correct name of the bot crawerer before but i will try it... thanks
Please note that, such ridiculous crawlers do not honor robots.txt rules. So you can use htaccess rules to deny access from such bots. I am quoting a few lines from a recent entry from our blog.
Following is a sample .htaccess rule, that will help you to block specific bots from accessing your website.
SetEnvIfNoCase User-Agent “BOT” bad_agent
Deny from env=bad_agent
Please note that you should replace “BOT” with corresponding BOT name. For example, the following lines shows access from Bing Bot and Baidu.
Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)”
Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)”
So to block the above two bots, we can add the following lines to .htaccess file of the respective website.
SetEnvIfNoCase User-Agent “bingbot/2.0″ bad_agent
SetEnvIfNoCase User-Agent “Baiduspider/2.0″ bad_agent
Deny from env=bad_agent
thanks for that info really appreaciate that.. I wonder though would that be the same result with the rewrite with regard to my original question. What i mean is that if the rewrite denies them, that would still take up server resources if they constantly run the script as they are now.
If so then which would be more efficient and take up less resources, the rewrite or the just ip deny.
It has been hitting my site now for almost two weeks every day all day checking for file names and directories.
On a dif note just fyi, i did put WHT as a resource here cause i appreciate what you all do.
Both options will take the same amount of resources as both will return 403 code. Make sure no HP pages are needed to serve for 403 error if you use either way. Unfortunately there is nothing other you can do from your end to stop bots as your access is limited on a shared server. Ask your host to block it in server level or at service level and thus prevent reaching it into your website. I believe they will assist you to block bots like "Baidu" as it is one of the horrible bots.