Results 1 to 5 of 5
  1. #1
    Join Date
    Jul 2006

    googlebot...30+ GB bandwidth per month?

    I recently checked my stats page and noticed an anomaly-googlebot is consuming about 30GB of bandwidth on my site per month.

    if I wanted to block googlebot from, how exactly would I do so? What would robots.txt look like, and where would I put robots.txt, in the directoryA directory?

  2. #2
    Join Date
    Aug 2005
    If it's just googlebot I'd setup a google site map.. That way you can tell it how often you expect the content on each page or area to be updated. Then you don't waste any bandwidth.

  3. #3

  4. #4
    wow thats a lot of bandwidth, make robot text file and put it in dir as files, i still cant belive it ate that much though amazing

  5. #5
    For me it consumes even more . But google bot is always welcomed in my site

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts