don't know if this is the right place to post this...(hope it is).
I've got a few colocated servers hooked up to a 3COM switch. When I looked at the MRTG for the ports, I noticed that all the ports had a similar inbound traffic graphing. Thinking that something was amiss, I sniffed the port and found that I was seeing a lot of traffic from different subnets (could even see pop3 passwords!).
Now, I contacted the IDC and they said that this was normal, and that that was how their network was configured (I did tell them that other IDCs didn't have this issue). I think something's not right somewhere. And assuming that the IDC won't fix this issue, how can _I_ fix this issue? I don't want to have constant 1Mbps of inbound traffic that I don't even use.
Now, I contacted the IDC and they said that this was normal, and that that was how their network was configured
They shouldn't have their network configured that way. A proper network configuration would give private vlans to each customer, and you should not be seeing broadcast traffic for other customers.
Even more disturbing is that you're seeing actual data, and not just ARP broadcasts. Even without separate vlans, a switch is supposed to learn where a device is. That's what makes it a switch and not a hub. If the switch is sending non-broadcast data for an active device to all ports, then the MAC table is filled up or there's some other problem.
Yes, it is definitely disturbing to see data packets - especially passwords! They definitely have switches, so I'm guessing that something is definitely wrong. What can I tell them? Restart their switches? But that will only go so far until their MAC table is filled up again. Argh!
Better yet, look at the destination IPs of the POP packets, find the hoster who has those IPs, and send them a copy of your packet dump, with their customers' passwords. That will get the problem properly raised as a security issue.
The cause of the traffic could be the use of a CARP or MS-NLB cluster, by the POP hoster. Both cluster methods use a pseudo MAC address, shared by all servers in the cluster. By default, no server in the cluster ever transmits a packet with that source MAC. So the switch has no idea where to send packets, and is forced to flood the packets out to every server in the cluster.
Some hosters used to use "public" and "private" switchports (e.g. Cisco's "switchport protected") to keep customer traffic separate. CARP and MS-NLB both defeat that kind of security.
There are ways to do both CARP and MS-NLB without this problem, but the real solution is for your colo to move to separate VLANs per customer.
I wonder if their switchport, to which your switch uplinks, was previously used as a SPAN port but nobody removed that config before giving the port to you. If so, that might explain why you were seeing (what might have been) all other traffic on that switch.