i have just been to google Analysis of cached robots.txt . in the test url against robot.txt file box i got my domain name. one without www.one (http://www.one) with www.

the one without www is detected as the directory and the www is not in domain. what does it all mean . i have no robot .txt file at the moment but want to have one once i find out how to use it. thank you.