illusionist
02-08-2003, 05:34 PM
I want to ban the following bots from all files on the server
psbot/0.1
(+http://www.picsearch.com/bot.html)
NPBot-1/2.0
(+http://www.picsearch.com/bot.html)
How do i setup the robots.txt file to ban these specific bots from scanning my site?
Currently its like this:
# For domain: http://www.iomx.com
# All robots will spider the domain
User-agent: *
Disallow:
psbot/0.1
(+http://www.picsearch.com/bot.html)
NPBot-1/2.0
(+http://www.picsearch.com/bot.html)
How do i setup the robots.txt file to ban these specific bots from scanning my site?
Currently its like this:
# For domain: http://www.iomx.com
# All robots will spider the domain
User-agent: *
Disallow:
