I always hear we have "multiple redundant connections" and I begin to wonder if these words are just made up for advertisement.

There seem to be so many outtages caused by network issues. I thought if one connection fails another one kicks in. Meaning you won' t have any downtime.

I also wonder how it technically works. What I imagine is that there are multiple connections going into one datacenter. One of those connections is used as main connection and the other ones are only for backup purposes.

If it works like this how can a company afford having so many connections lying around idle? Let' s take nac.net (I am not complaining about nac! I have only heard good things about Nac and they seem to be one of the most reliable datacenters out there!) as an example.

They claim to have 75 diverse providers. I cannot believe you can afford having 74 connections just for backup. Moreover I don' t think it is necessary to have so many backup connections. I think even in aviation there are only 7 backup systems for each vital function.

So what' s this all about? How does it really work? Why are there so many outtages?

NAC uses bgp. On one of our network segments we provide redundant connections by using a Catalyst 5509 as our core switch for that segment. We have a RSM (Route Switch Module) and Supervisor Engine 3 in it with two 24 10/100 modules. We simply create vlans for several ports i.e. 4/1, 4/2, 4/3, 4/4. 4/1 We would plug in our main connection to NAC, 4/2 Would be a connection to HE.net, 4/3 Would be a connection to Another provider and so on. We use floating static routes on our router so that if one route is down i.e. our primary route to NAC dies it routes all traffic over the HE.net connection. We then have some custom scripts that announce the routes for our IP blocks to route to the HE.net WAN IP on our router. Keep in mind this requires your own ASN # and your own IP blocks via ARIN. I doubt many other hosts do it this way but it works for us :)

Is NAC directly connected to those 75 providers? In other words, is NAC ON-NET on those 75 providers?

dektong,

They are connected both via Transit and Public Peering.

Check out this network map... http://www.nac.net/networkmap.asp

actually, I am aware of the network page, but I just never understand how to interpret it :) When we talk about "NAC Datacenter", it's the one in Parsipanny, NJ, right? Looks to me that the datacenter itself only get 3 physical lines going to it (2xOC3, 1xDS3). To my understanding, no matter how good the peering is, the datacenter itself is only directly connected to UUNet (via DS3), and two other providers (unknown) via OC3.

I have been contemplating NAC network page for quite a while, just not sure how to interpret it. But to it does not seem to me that the datacenter is actually ON-Net to the 75 providers they claim, does it? In short, if all of those three lines (2 OC3 and 1 DS3) are down, then the whole network will be down (no matter how many providers they peer), right?

you always have to read it carefully, because a lot of times they are talking about their backbones. Like a hosting company could say they are on Multiple OC-3s when technically they are, but through their 56k to their ISP. a little extreme, but you get the point.

(a little off topic, but deals with connection advertising)

You are correct in that the facility is connected to UUnet, and two of their other POPs/DCs.

As a matter of fact, I believe this topic was covered earlier comparing them to InterNap.

I actually just posted something about this in another thread somewhere. Anyway, I am a BGP engineer, and routing is what I do for a living. I've found that the vast majority of companies claiming "multiple redundant connections" are actually referring to their upstream provider. There are very few providers that actually have the knowledge and expertise, as well as the resources to do it themselves.

There is a very easy way to tell if someone really is multihomed or not. Telnet to a route server of choice (I like route-views.oregon-ix.net because it receives full views from many different providers) and do a "sh ip bgp [ip]" and see what AS# is announcing one of the provider's IPs. Or do a traceroute from that route server, each hop will show the AS number that is announcing that particular IP. You can look up the organization owning an AS at arin.net.

Grant

Hey Grant,

Great info. I always used the site http://www.dca.net/support/lg.html or similar looking glass sites.

One question I tried telneting to the server you gave, and it said we had 50 available paths. I must be honest, we do not have 50, we have 2, soon to be 3. Is that 50 pretty much a cumulative of all the possible paths that each of our providers have? Kinda like what you guys were discussing where people advertise having the same amount of paths as their upstream?

Regards,

That particular router actually peers via eBGP-multihop with a bunch of different networks. It's used mostly for debugging purposes, so that engineers can check to see how their AS is seen by lots of external ones.

If you do a "sh ip bgp sum" on that router, you'll see all the sessions they currently have active. You'll see ~50 paths to your AS, as each of those sessions represents a network that's peered to the route server, each having its own preferred path to your AS.

Grant