Web Hosting Talk







View Full Version : unexplained bandwidth usage, mrtg


atr
11-20-2002, 02:12 PM
I recently began colo for a server that I built. When I first checked mrtg bandwidth usage, I was surprised to see over 100 kbits/second, even with nothing running on the server. I tried rebooting to server, to see if bw usage would dip--it didn't. Then, I shut down the server. Still, mrtg showed 100 - 150 kbits/ second, almost all of it outbound from the switch.

MRTG is still showing significant bandwidth usage at all times. Here's a recent chart:

http://www.22u.com/files/mrtg1.gif

As you can see, it's pretty crazy. BWM on the server shows almost no usage--about 0.5 kbytes/second total.

The only explanation I can think of is that another server on the switch is putting out a hell of a lot of broadcast traffic. Anyone have a better explanation?

nwtg
11-20-2002, 02:57 PM
Originally posted by atr

The only explanation I can think of is that another server on the switch is putting out a hell of a lot of broadcast traffic. Anyone have a better explanation?

That could certainly be, however I would do a Top from root to look at any background processes that shouldn't be, and "kill -9 pid" if there's something that looks off.

If this is recent, check your firewall/ipchains and make sure you're not being used as a means to run a synflood to some other boxes on the rack.

The Prohacker
11-20-2002, 03:11 PM
Also.. Make sure the kernel and any other software is up-to-date.. If your server is only pushing 100k, I kinda doubt its been compromised and turned into a pubstro but its possible...

cbtrussell
11-20-2002, 03:19 PM
Looks to me like they're showing you stats for the wrong port.

atr
11-20-2002, 03:22 PM
I don't have any reason to believe that the server has been compromised. In any event, mrtg shows over 100kbits incoming (i.e. outgoing from the switch) even when the server is powered off!

What sort of programs on other servers would broadcast this much volume (over 1 gig per day)?

atr
11-20-2002, 03:24 PM
Originally posted by cbtrussell
Looks to me like they're showing you stats for the wrong port.

That's what I thought at first, of course. But, I can get mrtg to show spikes from wgetting uumap.tar.Z, etc., so it's defintitely the right port.

silversurfer
11-20-2002, 04:14 PM
did it cross your mind to ask your coloc. center?

atr
11-20-2002, 04:20 PM
Originally posted by silversurfer
did it cross your mind to ask your coloc. center?

Yes. They have provided no explanation, but tell me they're looking into it. However, that was a few days ago, so I am getting impatient (understandably, I think).

silversurfer
11-20-2002, 04:28 PM
do a few things...

ps -A check what processes are running and see if there's any that's suspicious and unaccounted for.

netstat: check connections to your computer. See if there's any suspicious connections that you can't explain.

As for the Colocation center, I would still bug them incessently

atr
11-20-2002, 04:44 PM
Originally posted by silversurfer

As for the Colocation center, I would still bug them incessently

I think this is the only option. Thanks to mrtg, I can see that it's affecting everyone else who's on the same switch. . .

RutRow
11-20-2002, 05:20 PM
Just run tcpdump. 100kbps should be pretty easy to spot.

TomK
11-21-2002, 12:00 PM
Wow...

Anyhow, we found the issue and have identified a solution, it was a small broadcast storm based on a few servers being multi-homed to different switches, and linux being broken the way it handles this. The switch just started sending packets for these servers to every port on the switch.

Waiting to reconfigure the clients servers and the way he is multi-homed.

And as I mentioned to you atr, you will not be charged for any traffic this month, so go transfer something!

Tom

Rebies
11-21-2002, 02:44 PM
atr: can I download something? ;)

atr
11-21-2002, 04:35 PM
Originally posted by Rebies
atr: can I download something? ;)

Yes. I will be offering file mirror/download hosting. 100 mbps port, up to 60 gigs disk space, and all the transfer you can push before December rolls around for only $50! PM me for more details.