Results 1 to 23 of 23
  1. #1

    Angry Limelight - Ugh!

    As some of you know I work for a major ISP which will not be name. Here is a little story about one of our vendors and their chosen DC/Network provider. So one of our vendors for one of our high bandwidth products uses Limelight for colo and their network.

    Today speeds from this vendor dropped from the usual 350 KB/s to 2 KB/s. I immediately started a support issue with our vendor on this issue as this was majorly affecting customers. The vendor in question then started a support call with Limelight on the issue and the trouble shooting process began. Well after a little over an hour they realize what was wrong. Limelight Leguardia (sp?) actually ran out of outgoing available bandwidth (I'm not joking). They seriously ran out of outgoing capacity which resulted in 2 and a half hour customer impact while they re-routed traffic. This was after an extra impact this morning to customers of 44 min because the foundry code they loaded blew out the VIPs they had setup.

    Now normally I wouldn't say anything but this is definantly not the first time (or I'm assuming the last) that these guys have blown it. Most of their network has single points of failure where a single line card holds the entire thing together, so whenever they do a firmware upgrade on that line card they have to take down the entire network.

    Our vendor also has equipment in the PHX (Phoenix) pop and one night connections between LGA and PHX became a problem. You could transfer at 5 Mb/s from either pop but when you transfered between the two you got around 50 Kb/s (it went over limelight's wvfiber transport). The Limelight CTO was contacted and actually stated that this was not a problem what so ever.....

    All in all I have never seen a DC/Network perform so poorly in my life and I would never trust anything important with these guys.

  2. #2
    Join Date
    Mar 2003
    Location
    New Jersey
    Posts
    1,277

  3. #3
    Join Date
    Aug 2002
    Location
    Seattle
    Posts
    5,512
    Limelight certainly has its issues. Not to say that any of this is incredibly bad, but it's not what you would expect for such high end pricing.

    In any case, the network (for what it's worth) is probably the one thing they have going for them (the quality of support certainly is not).

  4. #4
    Join Date
    Feb 2004
    Location
    Louisville, Kentucky
    Posts
    1,083
    Originally posted by NSCNAP Jeff
    In any case, the network (for what it's worth) is probably the one thing they have going for them (the quality of support certainly is not).
    The unprofessional manner in which they conduct business is enough reason for me to stay away from limelight. I'm sorry to hear that RossH is forced to deal with those jokers.
    Jeff at Innovative Network Concepts / 212-981-0607 x8579 / AIM: jeffsw6
    Expert IP network consultation and operation at affordable rates
    95th Percentile Explained Rate-Limiting on Cisco IOS switches

  5. #5
    Join Date
    Jan 2005
    Location
    San Francisco/Hot Springs
    Posts
    988
    I think that would probably have me searching for a credit or another provider.
    Heck, nothing wrong with multihoming your network these days....
    AppliedOperations - Premium Service
    Bandwidth | Colocation | Hosting | Managed Services | Consulting
    www.appliedops.net

  6. #6
    Originally posted by wotanworks
    I think that would probably have me searching for a credit or another provider.
    Heck, nothing wrong with multihoming your network these days....
    Unfortunantly only the Vendor could seek relief from Limelight, not us. We can't really seek credit from our vendor either as the person who signed our contract with them was apparently brain dead and thought 99.5 sounded like a good number for monthly uptime for an enterprise service which effects tens of thousands of people (Side Note: This is why marketing guys should never be allowed to sign technical contracts without a techies supervision and I'm not talking about the head of a techie department either as they usually know less then the marketing guy. If you own or are high up in a company make sure you set this rule or you company will lose it's *** [we have had to lose tens of millions because of this]).

  7. #7
    Join Date
    Aug 2002
    Location
    Seattle
    Posts
    5,512
    Originally posted by wotanworks
    I think that would probably have me searching for a credit or another provider.
    Heck, nothing wrong with multihoming your network these days....
    It's interesting you mention that. When I was hosting with them at PHX3 (their Tempe, AZ datacenter) they did everything they could to keep me from bringing in additional carriers short of telling me, "No, you can't do that, this is a captive facility."

    They did, however, give me the option of spending $2000/mo to buy a GE loop back to 120 E Van Buren on the condition that I not use it to BGP with nLayer (for whatever reason, I guess they don't like nLayer).

  8. #8
    Join Date
    Feb 2001
    Location
    New York City
    Posts
    385
    Originally posted by NSCNAP Jeff


    They did, however, give me the option of spending $2000/mo to buy a GE loop back to 120 E Van Buren on the condition that I not use it to BGP with nLayer (for whatever reason, I guess they don't like nLayer).
    What's not to like about nLayer? Good people. I'd certainly take nlayer over Limelight.
    Raj Dutt // CEO // Voxel dot Net Inc
    http://www.voxel.net // [email protected]

  9. #9
    Join Date
    Mar 2003
    Location
    New Jersey
    Posts
    1,277

  10. #10
    Join Date
    Nov 2002
    Posts
    2,780

  11. #11
    McDonalds McGriddles > nLayer > #NANOG Hosters > Limelight

  12. #12
    "Limelight - Making people want to stab themself in the face since 1995!"

    But seriously, I have talked to the guys at nlayer and I gurantee they are more knowledgable and friendly then the guys at limelight are.

  13. #13
    Join Date
    Aug 2002
    Location
    Seattle
    Posts
    5,512
    Limelight actually has some nice, knowledgable folks. Only problem is they're all at the bottom of the food chain.

  14. #14
    Join Date
    Jan 2005
    Location
    San Francisco/Hot Springs
    Posts
    988
    Originally posted by NSCNAP Jeff
    They did, however, give me the option of spending $2000/mo to buy a GE loop back to 120 E Van Buren on the condition that I not use it to BGP with nLayer (for whatever reason, I guess they don't like nLayer).
    Thats pretty amusing, I wonder how they would've enforced that.

    BTW, I'm a fan of nLayer, they're great guys over there.
    AppliedOperations - Premium Service
    Bandwidth | Colocation | Hosting | Managed Services | Consulting
    www.appliedops.net

  15. #15
    Join Date
    Aug 2002
    Location
    Seattle
    Posts
    5,512
    Originally posted by wotanworks
    Thats pretty amusing, I wonder how they would've enforced that.

    BTW, I'm a fan of nLayer, they're great guys over there.
    With a good old fashioned phone conference and a handful of management standing by stating something to the effect of, "You're not going to use nLayer, are you?"

  16. #16
    Originally posted by NSCNAP Jeff
    With a good old fashioned phone conference and a handful of management standing by stating something to the effect of, "You're not going to use nLayer, are you?"
    hey Mr NSCRAP , whats the deal with your network and the reverses for your interfaces?

    4 ash-bb1-geth7-3-6-0.telia.net (213.248.88.29) 2.470 ms 2.532 ms 2.358 ms
    5 iad-ord.ge21.core01-iad.nscnap.net (69.26.160.249) 46.327 ms 46.173 ms 46.246 ms
    6 69.26.160.218 (69.26.160.218) 45.729 ms 45.665 ms 46.065 ms
    7 dfw-ord.ge03.core01-dfw.nscnap.net (69.26.160.217) 45.992 ms 45.572 ms 45.882 ms
    8 NetSentryCorp.demarc.cogentco.com (38.112.21.70) 46.128 ms 45.813 ms 45.962 ms
    9 noc.net-sentry.net (69.26.161.252) 46.003 ms 45.800 ms 45.877 ms

    looking at this traceroute, first i hit IAD-ORD then some odd ip, then DFW-ORD, and then a cogent ip? wassup with that mang? which path am i taking, IAD-ORD or IAD---DFW----NRT----SYD----LAX---ORD? cmon!

  17. #17
    Join Date
    Aug 2002
    Location
    Seattle
    Posts
    5,512
    jimmytimbo:

    I do not work for NSCNAP anymore, that company was sold to NR Software who is in the process of revamping the network. I'm personally no longer involved.

    To answer your question though, that path is Ashburn, VA -> Chicago, IL -> Dallas, TX. You're reading it incorrectly: "dfw-ord" is the identifier for the Chicago to Dallas transport loop.

    Cogent is showing on the traceroute because the IP 38.112.21.70 is a Cogent IP which happens to be bound to NSC's Dallas core. It has nothing to do with Cogent, the powers that be are just being lazy and not setting up seperate interfaces for the Cogent uplinks and the local transport (and if you know anything about using a traceroute to view a network you would know that Telia is actually the transit provider that was being traveled prior to you hitting NSC's backbone.)

    I have no idea what NRT or SYD are nor do I know why this is being mentioned in a thread about Limelight

  18. #18
    are you sure the real reason for the cogent IP being visible is that the riverstone boxes respond to traceroute out their egress interfaces? in other words, while nscrap is using Telia for inbound traffic, they are using cogent for 0/0 while tunneling traffic across them!

    how much was nscrap sold to NR SOFTWARE for?

  19. #19
    Join Date
    Feb 2004
    Location
    Louisville, Kentucky
    Posts
    1,083
    Originally posted by NSCNAP Jeff
    To answer your question though, that path is Ashburn, VA -> Chicago, IL -> Dallas, TX. You're reading it incorrectly: "dfw-ord" is the identifier for the Chicago to Dallas transport loop.

    Cogent is showing on the traceroute because the IP 38.112.21.70 is a Cogent IP which happens to be bound to NSC's Dallas core. It has nothing to do with Cogent, the powers that be are just being lazy and not setting up seperate interfaces for the Cogent uplinks and the local transport (and if you know anything about using a traceroute to view a network you would know that Telia is actually the transit provider that was being traveled prior to you hitting NSC's backbone.)
    No, the iad-ord.ge21.core01-iad.nscnap.net hop is either named incorrectly, or is the same logical interface as is used for Telia transit. The original poster is not wrong. Either the NSCNAP DNS is wrong, or the network topology is braindead.

    I have no idea what NRT or SYD are nor do I know why this is being mentioned in a thread about Limelight
    Those are designations for airports in Tokyo and Sydney. Obviously the poster was poking fun at NSCNAP.

    To touch on another issue, though, you mention that you no longer work for NSCNAP. NSCNAP acquired your company, BLCC, as I understand. Can you say whether or not they sold BLCC back to you, etc.? Also, you mentioned that you were unhappy with Nishant at NR Software in particularly, and that was your motivation for moving from their facility to XO. Is that true, or was this really a precursor to divesting BLCC customers and assets from NSCNAP?
    Jeff at Innovative Network Concepts / 212-981-0607 x8579 / AIM: jeffsw6
    Expert IP network consultation and operation at affordable rates
    95th Percentile Explained Rate-Limiting on Cisco IOS switches

  20. #20
    Join Date
    Aug 2002
    Location
    Seattle
    Posts
    5,512
    Jeff,

    Nishant owns most of XO at 600 W 7th and I currently buy my space and bandwidth from him. To say that i'm unhappy with him wouldn't make much sense.

    The move was a result of Shaun being unhappy with the 200 N. Nash location in particular, due mostly to the increased costs at that facility. I'm impartial to either, but I just so happen to be located at 600 W. 7th now as luck would have it

    Divesting BLCC was a result of Shaun's personal agenda and had nothing to do with re-locating from 200 N. Nash.

  21. #21
    Join Date
    Aug 2002
    Location
    Santa Monica
    Posts
    125
    I thought I'd chime in here...

    NR Software is the owner of ASN 27524. We are in process of assisting Shaun in tearing down the network and integrating it into our network (due to financial reasons for him).

    We did not "purchase" NSCNAP. We are just re-aquiring some of the assets that we owned previously + purchasing any assets that we might find valuable.

    If anyone has any issues with the Xeex network / backbone (it has nothing to do with NSC's backbone), please ask and we will be more than happy to provide you information.

    As for the questions that we see frequently, I hope this clarifies things:

    1) NR Software is the parent company for Xeex Communications and ColoMAX.

    2) ColoMAX is our managed services and colocation division.

    3) Xeex Communications is our transit/transport services that we sell.

    4) NR Software is a software development entity that has been around for 20+ years.

    In the future we will be integrating ColoMAX and Xeex under just Xeex Communications.

    If you have any further questions, call us at 310-309-3715 x1, email at info [at] nrsoftware.com.

    Have a wonderful day!

  22. #22
    Join Date
    Nov 2002
    Posts
    2,780

  23. #23
    Originally posted by NSCNAP Jeff
    Limelight actually has some nice, knowledgable folks. Only problem is they're all at the bottom of the food chain.
    Considering the services they provide I would sure hope they have knowledable folks!
    data center directory - The comprehensive data center resource. Version 2.0 coming soon!

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •