Web Hosting Talk







View Full Version : Search Engines, robots, spiders


BooBoo
08-20-2002, 04:29 PM
The Vice-President of our company just contacted me in a Chicken-Little mode. She was upset that we were "not" being indexed by the search engine spiders, robots, etc. I asked her the obvious question, How did you determine that? She went to a search engine conference and picked up some software from Bruce Clay, LLC (http://www.bruceclay.com). It did a spider search on a virtual site and returned a 302 error, that the site was being redirected.
I tried to explain that we were being indexed and that this report did not really mean anything. Yes, Cobalt redirects the sites, but, that the spider would check a different way for the site. We should be fine.

Well, my explanation was not good enough. Any help, suggestions, or articles that I can use to calm her down?

lighty
08-21-2002, 12:54 AM
Hi,
While submitting the URL to the search engine you should take that metatags have been correctly entered into. Is it the problem with all the search engines or any specific one?

dreamrae.com
08-21-2002, 01:03 AM
i wish google would hurry up and spider my ass...:bawling:

dutchie
08-21-2002, 08:54 AM
BooBoo,

You could show her the entry of the company in the searchengines by typing the companies name, it least it shows her it is indexed.
Another argument is to find a side wich is listed on top but still shows the error you are talking about.

if you are unable to do one of those things, she's probably right :D

BooBoo
08-21-2002, 11:47 AM
I spoke with a person at the Bruce Clay company, and I really think that this is a non-issue. What I am trying to determine is how the spider works. If it accesses a server by ip address, then how does it know to index the other sites?