Results 1 to 4 of 4
  1. #1
    Join Date
    May 2002
    Raleigh, NC

    Detecting and blocking web bots

    We've seen an increase in malicious web bot activity targeting sites with WordPress and a few other popular CMS systems. Normally this isn't a problem, but some sites in particular get pounded hard enough that the bad bot requests eat up a lot of server resources.

    For the bigger and longer-running attacks we've been implementing various ad hoc approaches on servers such as detecting/blocking bad IP addresses, bad query patterns as well as specific user-agent patterns.

    I'm trying to take a step back and come up with a more comprehensive and long-term approach to malicious web requests. Any other input from people responsible for administering lots of these sites or high-traffic sites would be appreciated. Looking for info such as;

    - are there any reputable centralized databases of IPs that are malicious yet safe to block (sort of similair to spamcop/spamhaus)
    - are there any companies/organizations that publish malicious attack patterns with very low chance of false positives? Has anyone used Trustwave Modsecurity Rules, Sourcefire / Snort VRT rules, or similar services for detecting malicious web traffic?

  2. #2
    Join Date
    Nov 2001
    Ashburn, VA
    You can use mod_security to block bad bots. I'm a big fan of ASL. Their modsec rules are pretty much the best you can find, and the rest of the tools in the suite make a great package.
    Affordable web hosting, design, & domain registration services since 2001

  3. #3
    mod_security with a good ruleset is definitely the way to go. There are some free rulesets you can find out there that are updated fairly often that are just as good as ones you would pay for (if not better).

    Anything else would require an IPS of some sort which means additional equipment.

    Some webhosts actually include IPS protectoin, so if moving is an option you might want to consider that. It's possible you've outgrown the current host.

    If you don't want to go either route then iptables blocking is the way to go, as tedious as that is.

  4. #4
    What web server are you using?

Similar Threads

  1. Is my code for blocking ip and bots correct?
    By sash_007 in forum Programming Discussion
    Replies: 8
    Last Post: 04-10-2010, 01:13 AM
  2. APF and auto-blocking bots
    By Adam A Flynn in forum Dedicated Server
    Replies: 13
    Last Post: 04-14-2006, 12:00 PM
  3. Blocking IRC Bots
    By DigiCrime in forum Hosting Security and Technology
    Replies: 13
    Last Post: 09-29-2004, 12:08 AM
  4. Blocking Bots from logging into forms...
    By XYPHEN in forum Hosting Security and Technology
    Replies: 3
    Last Post: 01-16-2004, 08:31 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts