Robots flood my server...
I use Outpost Firewall to view active connections to my server. If I don't restart the httpd service on a regular basis my server will grind to a halt from being flooded by robots.
I currently have the service set up to restart at Midnight and Noon every day. Sometimes that's enough, lately it's not. For example, I checked an hour ago and I had 385 connections to httpd. At least 50% of the connections were robots - tons of the same IP addresses and they're just crawling the site.
Almost all of the connections show up as less than 1kb bytes received and 0 bytes sent per connection.
I already have a good 20 connections by these robots and the connection time shows as 11 minutes... I just browsed to a web gallery page on my site figuring that'd be mildly "intensive" on connections with all the thumbnails and my connections aren't lasting more than one minute.
So, what's with all these connections that are lasting 10+ minutes? I've even got one connection that has an Uptime of 30 minutes, bytes sent 65811, bytes received 180. It seems like something with these robots doesn't terminate correctly...
Any ideas on what to do so these connections quit jamming my server up? It's like a very very slow DOS...
Windows 2008 R2 SP1 x64
Core2Quad Q9650; 8GB ddr2