I'm wondering if anyone knows of any fairly effective methods/techniques of scanning a fair amount of hosted dedicated servers for copyrighted software. How do other big hosts out there attempt to keep this sort of thing under control (illegal distribution of copyrighted software).

I don't think there is a real way to watch for this kinda stuff. What stops the user from just naming everything blah.dat or whatever? I think some hosts just watch to see what kinda connections are leaving there network (port wise maybe?) and just wait for reports from people :)

For dedicated servers, what would stop the end user from turning off/removing whatever you installed to try to watch them?
Unless you have the program listen on a port, and then monitor the port (and kick arse when the port shuts down) then I honestly can't think of any solutions for such a thing.

There are a few hosts I saw a year or so ago, that just didn't allow zip files on any of their servers, just to try to stop warez and such. They didn't last long :p

~Francisco

You can easily look for files such as .rar .zip .mp3 whatever you'd like.

One of 2 ways:

locate zip |grep home >> somefiles.txt

or find /home |grep zip >> somefiles.txt

Those are some examples of how to look for what you want to find

maybe they monitor the amount of traffic that comes out of the site and from which files.

For example, if a site is only 1 gig in size, and about only 20 megs are for the html page, and the other files get more hits than the webpages/image files then that could be suspcisious...

i wonder if there is also methods of running an ftp site and encrypting the entire directory structure (or maybe storing everything in one huge file?) to avoid people using find and locate.

Originally posted by JSH-John
I'm wondering if anyone knows of any fairly effective methods/techniques of scanning a fair amount of hosted dedicated servers for copyrighted software.

Why would you want to do that? Reduce bandwidth usage or?
Remember that you are not liable as long as you comply with the DMCA.

Its not really for either reason. I'm not worried necessarily, but I'd like to be able to make sure people aren't abusing our services. Ya know?

Originally posted by JSH-John
make sure people aren't abusing our services

What sort of abuse are you referring to?

I'm not defending piracy; I'm just wondering whether it makes any business sense to bother with this.

Energy,

Using there accounts to store/host piracy would be an example of abusing there servers,
as there TOS would no doubt state.

Regards,
Thomas Currie

ThomasC, sure, their TOS would state that but that does not answer the main question. There must be more to it then "You are abusing the service simply and only because you are doing what I told you not to."

I'm looking for a reason to spend time and resources hunting for these files; is it worth it? It's not a legal reason and it is also not to save money (JSH-John said it's not about bandwidth). And unlike spam, this would not have any impact on other customers.

Is there a valid business reason to do this?

Contact Webdude via the PM here and ask if he is still offering assistance with his Warezhunter.
We now are testing our own special program called WarezHunter. It detects every method warez uses, and deletes their files automatically. It is something no other free host has, and is one Warez can't hide from! See it's dynamic updates below.

dont know if i trust a program that delete files automatically...

Funny, what I've found is most people that try to host warez, mp3's etc don't bother renaming them, they just assume nobody will ever look. And some don't, we catch people all the time with these sorts of things on clients servers.

Catch of the week? A reseller hosting company on a resold server with an entire directory labelled....... Hacking Tools. Now that's just asking to be caught lol.

Don't you as a host become responsible under the DMCA once you start actively scanning for warez, instead of just responding to reports?

sudo find /path/to/home/directory/* -name '*.zip' -or -name '*.rar' -or -name '*.bin' -or -name '*.BIN' -or -name '*.cue' -or -name '*.CUE' -or -name '*.m*' -or -name '*.ace' -or -name '*.001' -or -name '*.r0*' -or -name '*.r**' -or -name '*.ZIP' -or -name '*.cue' -or -name '*.par' -or -name '*.exe' -or -name '*.M*' -or -name '*.A*' -or -name '*.R*' -or -name '*.I*' -or -name '*.E*' -or -name '*.iso' -or -name '*.nrg'

i use this shell command to find mp3's and warez, it's very cpu intensive...

this command is much smaller and is used to find only one type of file

sudo find . -type f -name \*.mp3 -ls -exec rm -i {} \;

I find it's always a good idea to know what type of files are on your box legal or not. It gives you perspective on who you're dealing with.

I've said it before many many times, the moment you begin to audit your users files you become liable for those files. Is this something you really want to put on yourself?

As web hosts it's our responsibility to find and remove harmful files on our servers. Not only are these files used to harm other people's servers but they could also be used to violate the security of the machine they are on thus risking the sites of other clients etc on that server

It's not like we root around in people's home folders, just run a few simple searches and it all pops up. I personally consider it negligence on the part of any sysadmin who fails to find and remove any user with warez files on their accounts.

What good is all the security precautions we sysadmins take if we're not even going to find and remove abusers on our own servers. Turning a blind eye to it isn't the answer.

Our clients know full well we search our servers for such files (and none object, they are happy we keep a close eye on things), if they aren't hosting them they have nothing to worry about, if they are hosting them, then they won't be there long.

The problem isn't becoming liable for the files, the problem starts when you find them and fail to remove them.

I dont' hesitate to run these searches, and I've yet to be sued for booting a client who hosts such things.

Donna, your confusing 2 very different beasts and I'm not saying turn a blind eye to either of them.

Warez != hacking, exploits or other malicious code, it's copyrighted material, be it songs, movies or software etc.

You should not need to scan the file system to identify potentially malicious binaries, this should be done at a network level where you have a greater chance of identifying the contents of the files.

When you scan the file system directly you're basically hoping that the user has not changed the name of the binary to something like hmm say, index.html. yet this index.html is actually a compiled exploit binary. A network level IDS will identify this and flag it for the admin to take action. This method of moitoring relieves you of the responsibility that is implied if you actively scan the host file system.

Sure once in a while we'll check to make sure a site is not doing anything dodgy based on odd traffic patters, system load, IDS sigs etc, but it is certainly not a policy we employ.

We have several methods of searching for these files, I was only mentioning the basic searches as one way to do it quickly and easily.