Results 1 to 11 of 11
  1. #1
    Join Date
    Jun 2011
    Posts
    61

    Help: Spider/Spam bot(s) Killing My Server!

    I have a "Hybrid" host with 2GB memory running on Centos5.6
    I am running apache with PHP as DSO + cpanel.

    I already equipped the server with CSF/lfd and for my main site i am also using Cloudflare where i am blocking China.

    Almost every week (especially on the weekends) usually around 8am my time (Europe) some spam bot is "attacking" my server and opens MANY apache tasks at once, eg.50+.

    This eats up all my memory and literally kills my server until i get up later and manually reboot from SolusVM interface. The server can be down for 8+ hours.

    I am fighting with this problem for some time already and thought csf/lfd and cloudflare should have solved it, but nada.

    I am also running all kinds of cache/optimization plugins on my sites which are running on Wordpress to reduce load etc, but as soon as this bot appears it simply overpowers my server with all those apache tasks.

    I am a point where i NEED a solution and i am not sure which one to choose.

    * Alternate web server to reduce memory consumption?

    I already did testing with alternative web servers (Nginx, Varnish etc.) to reduce memory consumption of the server but overall did not see any improvement, overall the memory consumption is the same. Work --> benefit ratio of exchanging apache for nginx is not there, IMHO. (Plus incompatibilities etc.)


    * Upgrading server with more ram?

    The most obvious solution could be simply giving the server 2GB more ram...problem here i dont know whether this would really solve the problem. If the bot does not appear, all my sites run flawlessly on the given hardware. I do not want to spend even more on the server/month if the added memory wouldn't even solve the problem

    * Software watchdog?

    I think a feasible solution would be a software watchdog which could reboot my server if it sees that apache etc. is down(non responsive for some extended time.

    Is there no such option anywhere already out of the box with Centos/csf? I am surprised since csf/lfd gives me all those alerts per email...is there an option to let it automaticaly reboot the whole server?

    What about this "softdog" application i just read about, would this be an option?

    Thanks!

  2. #2
    Join Date
    May 2002
    Location
    Moscow
    Posts
    1,602
    Hi,

    You could use csf or monit to reboot apache or whole VPS when your load average or memory usage is high.
    TK Rustelekom LLC Dedicated server since 2002, RIPE NCC member, LIR

  3. #3
    Join Date
    Jun 2011
    Posts
    61
    Hello, could you tell me where in csf i can configure that?

    thanks.

  4. #4
    Join Date
    Aug 2011
    Location
    Vancouver, BC
    Posts
    165
    This almost sounds like the slowloris attack:

    http://ha.ckers.org/slowloris/

    It might pay to try and use it against your apache server from home to see if you're affected. Or just switch to lighty/nginx if you have the time
    vanVPS Hosting | Vancouver Canada KVM VPS | 100% Uptime SLA
    [100Mbps Port] [Custom High Performance Solutions] [Secure Remote Console]
    PEER 1's Fast Fiber Network | RAID10 | 2 x Gigabit Uplinks Per Node
    http://vanvps.com

  5. #5
    Join Date
    Feb 2012
    Location
    London, UK
    Posts
    82
    You can try using Litespeed instead of Apache.

  6. #6
    Join Date
    Feb 2010
    Location
    Worldwide
    Posts
    61
    Hi,
    Incapsula has a DDOS mitigation service as well you may with to try (and not as insanely priced as that other big DDOS mitigation service). Personally I'm a big fan of Cloudflare as well, though Incapsula tends to work better for enterprise level stuff IMHO.

  7. #7
    Join Date
    Nov 2010
    Location
    San Francisco, CA
    Posts
    901

    Post Hi,

    Quote Originally Posted by GeorgRauh View Post
    I have a "Hybrid" host with 2GB memory running on Centos5.6
    I am running apache with PHP as DSO + cpanel.

    I already equipped the server with CSF/lfd and for my main site i am also using Cloudflare where i am blocking China.

    Almost every week (especially on the weekends) usually around 8am my time (Europe) some spam bot is "attacking" my server and opens MANY apache tasks at once, eg.50+.

    This eats up all my memory and literally kills my server until i get up later and manually reboot from SolusVM interface. The server can be down for 8+ hours.

    I am fighting with this problem for some time already and thought csf/lfd and cloudflare should have solved it, but nada.

    I am also running all kinds of cache/optimization plugins on my sites which are running on Wordpress to reduce load etc, but as soon as this bot appears it simply overpowers my server with all those apache tasks.

    I am a point where i NEED a solution and i am not sure which one to choose.

    * Alternate web server to reduce memory consumption?

    I already did testing with alternative web servers (Nginx, Varnish etc.) to reduce memory consumption of the server but overall did not see any improvement, overall the memory consumption is the same. Work --> benefit ratio of exchanging apache for nginx is not there, IMHO. (Plus incompatibilities etc.)


    * Upgrading server with more ram?

    The most obvious solution could be simply giving the server 2GB more ram...problem here i dont know whether this would really solve the problem. If the bot does not appear, all my sites run flawlessly on the given hardware. I do not want to spend even more on the server/month if the added memory wouldn't even solve the problem

    * Software watchdog?

    I think a feasible solution would be a software watchdog which could reboot my server if it sees that apache etc. is down(non responsive for some extended time.

    Is there no such option anywhere already out of the box with Centos/csf? I am surprised since csf/lfd gives me all those alerts per email...is there an option to let it automaticaly reboot the whole server?

    What about this "softdog" application i just read about, would this be an option?

    Thanks!

    Are you blocking the IPs in the CloudFlare threat control panel for the spam bot?
    CloudFlare Community Evangelist

  8. #8
    Add this to your htaccess. You may or may not want to remove msie 6
    Code:
    SetEnvIfNoCase User-Agent "(Baiduspider|Beta|CrystalSemanticsBot|Deepnet\ Explorer|disco|DLE_Spider|Exabot|Firefox/2|Firefox/3|HuaweiSymantecSpider|Indy\ Library|Java/1.4.1_04|Java/1.6.0_04|Java/1.6.0_22|Java/1.6.0_29|Java/1.6.0_30|Java/1.6.0_26|magpie|MJ12bot|MSIE\ 2|MSIE\ 3|MSIE\ 4|MSIE\ 5|MSIE\ 6|New-Sogou-Spider|Ocelli|Powermarks|Sogou\ web\ spider|Spinn3r|suggybot|Wget|*******|xpymep|Yandex|yeti|YodaoBot| /)" bad_bot
    <Files *> 
    Order Allow,Deny 
    Allow from all 
    Deny from env=bad_bot
    </Files>
    ******* = wow rack without the space. About time they got removed, they slipped below the radar with spam for months

  9. #9
    Switch to mpm_worker or mpm_event with FCGI or PHP-FPM

  10. #10
    Join Date
    Jul 2002
    Location
    London, United Kingdom
    Posts
    4,455
    switch web-server software, or adding more RAM will simply delay the point you keel over and die by a few seconds.

    you need to be blocking the source of the attack, as well as talking to your upstream about it.
    Rob Golding Astutium Ltd - UK based ICANN Accredited Domain Registrar - proud to accept BitCoins
    Buying Web Hosts and Domain Registrars Today @ hostacquisitions.co.uk
    UK Web Hosting | UK VPS | UK Dedicated Servers | ADSL/FTTC | Backup/DR | Cloud
    UK Colocation | Reseller Accounts | IPv6 Transit | Secondary MX | DNS | WHMCS Modules

  11. #11
    Join Date
    Jan 2008
    Location
    Europe
    Posts
    779
    We saw significant performance increase using nginx with php-fpm.

    If theres a single IP opening 50 connections you can configure your firewall or httpd to block these. Also if the requests look similar you can use a script to automatically ban the IPs.

Similar Threads

  1. Help with spam (bot?)....
    By pasobuff in forum Hosting Security and Technology
    Replies: 4
    Last Post: 05-13-2011, 04:03 PM
  2. Happy with Host. But Forum Spam is killing me !
    By Vicente Duque in forum Web Hosting
    Replies: 13
    Last Post: 04-03-2006, 08:19 AM
  3. DNSpider.com (Domain Spider Bot and Tools Website)
    By CrazyTech in forum Other Offers & Requests
    Replies: 14
    Last Post: 08-28-2005, 12:19 PM
  4. spider crashes server
    By netserve in forum Hosting Security and Technology
    Replies: 4
    Last Post: 01-20-2003, 08:38 PM
  5. I really need help! SPAM is killing my business!
    By StarGate in forum Running a Web Hosting Business
    Replies: 36
    Last Post: 12-15-2002, 12:22 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •