goervin
07-27-2004, 02:28 PM
I am looking to put a links directory on my site. I have researched the different options and come up with two main ways to do it. Use html directory program such as Arelis or use a search engine database directory.
The reason that I am adding the links directory is to get more links and improve my search engine rankings.
My Question is: Do search engine spiders index data in a database. I have read many artical on the "Deep Web" that say spiders can not index database information.
e-zone
07-27-2004, 02:56 PM
as far as i know the spiders cant get information from databases that are dynamicly generated when visitors come to your site.
best bet is to get one that makes html pages or makes .html links without the use of javascript ;)
*correct me if im wrong :)
saghir69
07-27-2004, 03:20 PM
from what i hear some search eingines do spinder dynamic page and some don't. but thier are ways arround it(which i'm currently looking for) something called mod rewrite, whatever it is lol
Search engine spiders will follow links... they don't normally "index data in a database" as such, but will follow links that lead to dynamically generated pages which draw from a database. So as long as a user doesn't have to, for example, fill something in on a form to get to your data, it should be spidered. But avoid urls with more than one or two variable strings (that is, using characters like "?" or "&" excessivly). Search engines will give such urls lower priority and will avoid getting deeply into them, to reduce the chances of getting trapped being passed from page to page.
What saghir69 is getting at is that you can use mod_rewrite to "rewrite" those urls so that they have a more "standard" format.