Does anyone know of a php script that works with pgsql or mysql as a backend that you can feed domains and it will index them all kind of like google or something close? Also needs to be able to follow links from sites I give it like a syder.

Thanks

http://www.aspseek.org/ but i think you need your own server for it.

I like phpdig.

it can also index doc, pdf and a few more I think with additional software binaries.

good luck,
vark

Had to have 5 to post a offsite URL, so here it is below.

http://www.phpdig.net/