Hi, I have a phpBB forum and I always see google bots, MSN bots, and other search engine bots spidering my forums all the time. However there is a new bot from France with the IP of 212.27.41.31 and it has been on my site nonstop for about 4 days straight and there are about 20+ of this same IP being displayed as crawling my forums. Is this bot sucking up my bandwidth to a degree where it would be something to worry about, or should I just ignore it? I am thinking about placing this IP in my firewall if it is causing any harm. Thanks

the ip does reverse to crawl21.dir.com (=212.27.41.31).
www.dir.com is a search engine, so i would say let them spider your forum, unless your running out of bandwidth, then ban them

Ok thanks, how much bandwidth do bots usually use? I have a lot but just making sure.

I would figure the spiders would visit/archive most of the threads on the forum, and go from there. So, if my logic is right, that would mean that it would be just as if someone went on each page of every topic?

I would think that after the first visit, though, it would just take what's changed since then.

Or i could be completely wrong :D

~Francisco

most bots skip images/movies/etc they only get text and poll to see if the image exists.