Hi fellas,

I'm looking at this page,

http://www.robotstxt.org/wc/exclusion.html#robotstxt

I have no idea how to implement it.

www.mydomain.com/index.php

I want google to only look at index.php file...Nothing else..
Or just stuff in the www.mydomain.com directory (/home/username/www/) directory.

Thanks.

I don't think META tags work any more. You'll need to use robots.txt instead.

Italic Indicates Code!

Make a file called "robots.txt", in it put the following code:
User-agent: *
Disallow: /cgi-bin/
Disallow: /forum/

Upload it to your website's main directory

If you want to stop the robot from looking at different folders, just add them in the same format I used :)

If you just want to block a specific robot (e.g. googlebot) then have:

User-agent: googlebot
instead of
User-agent: *

Hope this helps you!

Joe

Joe said it all!

<<Signature to be set up in your profile>>

Thankyou very much!

No problem whatsoever!