Web Hosting Talk : Web Hosting Main Forums : Hosting Software and Control Panels : Search Engine

[abbas]

Hello all,

Does anyone know of any search scripts in simple perl (they are very easy to install!), that preforms like google.com / yahoo.com or excite.com.
Maybe something that indexes all sites submitted.

The search scripts that I have found on the internet do not index sites, and the output is not that good.

Something simple would be nice, maybe I might create one by scratch, but where would I begin?

Thanks in advance to those who reply.

[kunal]

You could find some Perl or PHP scripts at http://www.hotscripts.com or http://www.cgi-resources.com

[abbas]

Hey, thanks.

HotScripts.com has a lot of search scripts, but one thing: is it "good" to have search engine data in a database like mySQL?

Thanks a lot.
Regards.

[kunal]

Originally posted by abbas
...is it "good" to have search engine data in a database like mySQL?

Its much faster, and safer. A Dbase like MySQL is highly recommended.

[abbas]

Thanks, now i need to find a (cheap) host to support it.

I have been testing some of the scripts on my home computer (web server configured with perl 5.0), and they run great.

But, is there anyway to create a portal like yahoo? With email and chat and everything else?

Most of the search engines i downloaded require the user to input his/her site address. Does anyone know of any crawler/spiders that go and index the net? Something like http://www.google.com would help.

Thanks a lot for all of your help.
Regards.

[akashik]

Abbas,

You might find something by net searching for 'spider search software' or something like that. If I remember correctly there are a few spiders you can get for nothing.

As for indexing the net and storing it locally.. umm, you *do* know how big the net is don't you? Google has indexed over 600 million pages directly (with around 1.3 billion indexed in total). Your 'cheap webhost' is going to have a fit if you try to pull that into your database.

Maybe a better solution is a search engine script that reads from the databases of others. ODP++3 springs to mind as one I was looking at a while ago. (available via hotscripts). If I remember correctly it'll chache a certain amount of data it recieves to speed up searches, but doesn't bother trying to pull in a whole directory. It also connects to DMOZ which in my opinion is a better source to search from than most places. Also, it has a function that allows you to place links at the top of the listing, so if you know of a good site in regards to a certain search, you can artificially place it at the top of the results page.

So yeah, there are some good ones out there - just depends on how much time you want to spend setting it up.

Greg Moore

[abbas]

Hello (and thanks for the reply).

When you search on the internet, how much "relavent" sites come up? If I use other databases, I will get a LOT of junk, and users of my site will never find the relavent stuff.

I was thinking about the hosting solution.

I was going to use a server-colocation service to host the site.

What do you think?

And about the scripts: I have searched the net and all the scripts I have found cost money (a lot of money!). Free would be nice.

Thanks.
Regards.