Hi everybody,

I've got something interesting for you today.

This morning I had a call from one of our clients. He has a web hosting plan with us and he asked me why he had so many hits from someone with this IP address: 63.148.99.247

I investigated a little bit and found this IP on other users accounts logs. Strange...

Then I found this link -> http://www.aota.net/forums/showthread.php?postid=71568

Yes, this IP address belongs to cyveillance.com ! But why are they visiting so many of our servers?

I thought our servers were about to be hacked when I found another link on google -> http://members.tripod.com/~kenny_a_j/conspiracy/2arch8.htm (start reading from "Eyeball Diversion")

From that site: "It seems that they (cyveillance) have developed a technology called NetSapien, which is a smart web agent - a "spider" which searches the Internet for information by actually looking within sites and newsgroup postings, also managing to look at images and sounds."

Another quote "In other words, for a suitable fee, they will keep an eye on the web to see if anyone is trying anything nasty to discredit your company and/or steal your customers. Obviously, this has serious implications about freedom of speech. Potentially, everything I've ever posted on "Only Visible From the Air" can be monitored and checked for anything which could be damaging."

So what's going on? Because, not only cyveillance is checking (or spying) on users files but they are also stealing our bandwidth! Yes, bandwidth, this same client notice that in one day his site had recorded 17 meg in one day! The entire web site is only 1.25 meg! Well, not anymore...


Two things now:

1 - I think it would be a great idea if we (web hosting companies) could built up a list of Bandwidth stealers and intruders like cyveillance.

2 - We could also do something to strike back.


Any idea?



Vincent Hirth (CEO)
http://www.directoris.com
v.hirth@directoris.com

One other thing that I heard about recently. There are companies now that will sell "referral spam". That is they "visit" a page on a web site with a falsified referrer page. They sell this service for around $1000 per 100,000 sites visited with falsified referrers. This pays off for the person paying for this "service" in two ways. People looking through their logs tend to go visit pages they didn't previously know linked to them, so this is a form of "look at my page" advertising. Secondly, many web sites have a section where they list "people who link to us", which is often automatically generated from the referrer logs. This not only gets them a link on such sites, but search engines often score pages based on how many other sites link to it, so they get boosted SE rankings.

(cough) Mr Moore, the engineer who developed that NetSapien software aka "Internet Big Brother" used to work for Microsoft Corporation.

Well, better for Bill that this guy (Mr Moore) introduced his spy software under another name than Microsoft. But it doesn't mean that both companies, Microsoft and Cyveillance, are not related.

Here is my point: Does, Cyveillance, has legally the right to enter any of our servers, using our bandwidth by reading/getting files stored in it?



Vincent Hirth (CEO)
http://www.directoris.com
v.hirth@directoris.com

Umm.. Why dont you just learn how spiders work? If a spider does not use a Robots.txt file it is generaly considered harmfull and thus against any respecting Tier1 ISPs AUP which would make the end person stop, or loose their connectivity.
see
http://www.robotstxt.org/wc/robots.html

Robots can be a good thing for someone hosting a site SEVERAL search engines quietly run mild robots daily scouring the net eating small bits of bandwidth. With a Robots.txt file you can easily include and exclude certain pages of your site or your site completely

I'd suggest just dropping their IP's in your DENY list.

A list like SPEWS but for web servers?

a blacklist of ips/subnets to block from your web site rather than your email server?

Does this animal exist?

There aren't very many such outfits, so most have gone with just blocking those that cause them problems. Here's a list of some that have been identified as bandwidth hogs:

63.148.99.224 - 63.148.99.255 Cyveillance
209.73.228.163 webclipping.com
66.150.40.0 - 66.150.40.255 Internetseer

I don't have any experience with webclipping.com, but Internetseer uses up a lot of bandwidth "testing" sites, and then if at anytime due to any reason (i.e. even if they just pretend) they can't reach the site, they fire off a spam that basically says, "We were unable to reach your site. We are a web site monitoring service, we think you need our service, want to sign up?" This has caused considerable friction between hosts and clients, and it's much easier to explain that they couldn't reach the client's site because they were blackholed for being bandwidth pigs and sending spam.