Hi,

I am building a community web site. There are many scripts available on the web to retrieve content from other large web sites like Yahoo, or SlashDot.

Isn't it illegal to display those copyrighted content for commercial use? As in my case, I am looking at redirecting query and retrieving newsgroup content from deja.com. Since my web site will be commercial, is it legal to do that?

Thanks.

Jacky

in doubt... ask the owner of the sites. There is a high chance that they already have some way to do that that will suck less bandwidth and usage from their site and please you both (most likely if you put a link to them on your site)

And yes, it is illegal to use someone else's content and pretending it is your. It is illegal to redistribute someone's content without prior asking the permission.

Copyright © 2000 company. All redistribution, reproductions reserved.

That's what that mean :)

The only thing I'll add that it isn't illegal to *link* to another site. Many of these "headline grabbers" essentially just pull the headlines from the other site and automatically link (example: to yahoo). I doubt very much that permission is required nor copyrights are being violated just because you are using a script to generate the links.

exactly, sorry if that wasn't clear.
integral copy is not, linking is allowed.

try http://www.isyndicate.com

I'm wondering how to do a similar thing. What I want to do is have constantly updating news headlines from various sites, which link directly to the relevant article on the sources site.

I'm fairly proficient at html, but have a low knowledge of other languages. Is there any software out there to take out the leg work? I found some useful info on how to 'grab' excerts from other sites, but this would effectively be using the sites bandwidth - which may piss them off. What I would like to do is somehow get a search done ever couple of hours, store the info and then put it into a generated page. This should remove the problem of 'stealing large amounts of bandwidth'. It should also make the page download much quicker - I believe?

An example of a site doing something similar to the way I wish to do it is http://www.check4news.com
. Any ideas on how I can do this with relatively little knowledge of scripting? I've thought of moreover.com and isydicate but they don't have the sites I want to get headlines from.

Isyndicate.com is the best place for this sort of thing.
They have a huge breadth of content.

Look at http://www.vetaid.org/ for an example of how specialist some of their news feeds are!

Gordon

Unfortunately isydicate dont have the news feeds from the sites I require. I'm looking for sports news headlines for individual UK football clubs. I really think I'll have to do it myself. Is there no software out there that will make it easy for me? I'll have a go with scripts, but as I'm a novice I can't do anything to complicated unless I'm walked through it.

One excellent source of free news headline feeds -- they've got a pile of categories -- is http://www.MoreOver.com ...

I've seen php scripts, etc. for grabbing this info, I just don't know how on-the-level this is. Also, the ones I've seen are specifically for certain sites (example: to grab headlines from fresh meat). I don't know if it would be easy to modify the scripts or not (nor if it is on-the-level to do this in the first place).

I don't know how many of you know about this site, but I thought it was absolutely brilliant. This is the link to a php grabber script from the site, and the tutorial is brilliant - it also looks like you can adapt the script fairly easily to grab anything you want.

http://www.4cm.com/4cm/tutorials/4CM_Grabber/

However, if I use this it would seem that I would be stealing bandwidth, as a link to the source site is created when the page opens. I don't really want to steal bandwidth, as I would certainly not appreciate it if someone was doing it to me. - However, what occurred to me was that if I was only grabing about 10 headlines from a site would this mean that the only bandwidth I was taking was that required to download those headlines or the whole page. If it was only the bandwidth required to download 10 headlines then this would be tiny - <1k. I'm not really sure how it works though. Would it set up a permentant connection to the source site, while my page was being viewed, or would it just get the requested information?

The tutuorial is one of the best I've seen on the net, and well worth reading just for interest.

Your thoughts are very much appreciated.

Again, if in doubt, always ask the site owner and ask what the best scenario would be for all involved. It's most likely that either he won't mind and lets you go ahead, or he'll describe a solution that benefits both of you.

Maybe a word from the other end of the discussion would be of help.

On CodeBrain.com, we're very freqently asked, often by other resources sites, if they can quote sections of our content or copy, or sometimes entire tutorial articles.

We've generally never refused such a request -- provided that the requesting site is legitimate and reasonably well run; that it carries no objectionable (i.e., adult, hateful, discriminatory, or illegal) content; that its use of our content does not in any way injure or defame CodeBrain.com; and that the use of such content is not for the purpose of direct profit by the posting site.

We *do* require that proper and plain copyright attributions are provided with all such material, and that clear links to CodeBrain.com are given at the beginnings and ends of each segment of material.

(As a side note, we absolutely forbid the download of *any* CodeBrain.com applet or script from any site but our own -- that's something we will come after you for like the hounds of Hell, replete with lawsuits, because it injures us economically.)

In our view, the result of having another site carry sections of our content (within reason!) is that we benefit by a bit of traffic from the site carrying our content; and they benefit by having useful content for their visitors -- a "fair enough" arrangement.

Nonetheless, we would indeed by prone to considerable dismay, probably evidenced by a subpoena, if you committed the discourtesy of just walking off with our content without explicitly making permission arrangements with us.