Hi, I just found that my bandwith usage is almost 2GB in the last 24 hours.

There wasn't so much visitors in the last 24 hours, but a lot of crawling.

The most strange thing is that those bots/spider like crawling on my calender, I saw they crawl the calendar day by day!:eek: :eek: :eek: :confused: :confused: :confused:

ps: my bandwidth limit is only 6G/month!

create a file called robots.txt and put in your website's root directory: /
put this inside the file:

User-agent: *
Disallow: /


Now, the robots will stop visiting your website.

alternatively if you need your website indexed by search engines, contact them and tell them your site & issues and for them to slow down the crawl on your site.

i think the email for google was googlebot@google.com, i'm sure you might be able to find a email for msnbot and inktomi(yahoo) somewhere also.

Not sure if this is the best idea, but I only let bots spider the pages I want them to... use the robots.txt like mentioned above and disallow anything you don't think they need to know about. Like your calendar page. Any image or include dir doesn't need bots poking around, etc... you can save a bit of bandwidth there...

By the way, you can learn more about the robots.txt standard at http://robotstxt.org