Results 1 to 16 of 16
  1. #1
    Join Date
    Apr 2002
    Location
    Houston, Texas
    Posts
    842

    Inexpensive Network Solution?

    Hello,

    I am asking this question in order to get advice from a network technician as to what equipment I will need if I ever start a hosting business someday:

    If were to get collocation space in a carrier neutral datacenter, and get one burstable Ethernet connection to Level3, one to XO, and one to MFN, what equipment would I need in order to use all three of the connections? I am guessing a router of some sort, but what kind. Keep in mind that this will be a low budget design at first; I will not be buying any "big fancy six digit routers" What is the bare minimum that I would need in order to do this?

    I would like to have some sort of technology that routes the data according to the best link (I think this is called BGP), but again not necessary if this is very hard to setup.

    Can anyone recommend a under 1-10 thousand dollar solution? The lower the better, remember this is a startup solution

    Some people have also recommended just a standard Cisco 2924 switch, will this do the trick, or is a switch inefficient?

    Thanks, Kyle

  2. #2
    Join Date
    Feb 2004
    Posts
    390
    Bare minimum would probably be an edge router:
    1-2U server (low spec) w/ at least 4 10/100 eth. ports. Then you just need router software (many open source solutions).

    Finally you would need a switch of some sort (there are MANY out there...very cheap).

    Keep in mind this is the CHEAP CHEAP CHEAP route to go and is not the way to go if you are looking for stability.

  3. #3
    Join Date
    Aug 2002
    Location
    Seattle
    Posts
    5,512
    Here is a quote that I received not too long ago on a Cisco 7200 series router capable of BGP4 over 2xGigE interfaces, just to give you an idea:

    Cisco 7204VXR w/NPE300 Engine – C7200-I/O-FE – Single AC power

    2 x PA-GE single port gigabit modules – Latest IOS installed – Rackmount H/W

    Price………….$10,600.00

  4. #4
    Join Date
    Jul 2002
    Location
    London, United Kingdom
    Posts
    4,366
    lower costs does not always help even in the short term, if this is going to form part of your core infrastructure, getting it *right* is much better than getting it cheap ...

    >If were to get collocation space in a carrier neutral datacenter,

    good start

    >Ethernet connection to Level3
    >one to XO
    >and one to MFN

    if they'll present them as fibre rather than cat5 for only alittle more, then take them up on it ...

    >all three of the connections?
    >I am guessing a router of some sort

    good guess - a bgp4 capable router is what you'll need, in fact for redundancy you'll need two routers, as they *do* go wrong, and downtime will kill your business - hardly any point having multiple carriers if you still have 1 point of failiure ...

    i would recommend you look @ 4 * 1u servers, running bsd and zebra, one per transit, one spare to which you backup all the other bgpd.configs etc - again, removing the single point of failiure
    transit into to fxp0, out throuh fxp1 to a switch

    failing that you could go the tradition cisco 7206 router "route" but at considerably more cost ...

    for switches, the cisco 3550-emi series are very reliable and not too costly


    >I would like to have some sort of technology that routes
    > the data according to the best link (I think this is called BGP)

    ospf - open shortest path first

    >just a standard Cisco 2924 switch

    obsoleted a couple of years ago, and the replacement (2950) is underpowered for what you're expecting, the 3550's will do the job ...
    Rob Golding Astutium Ltd - UK based ICANN Accredited Domain Registrar - proud to accept BitCoins
    Buying Web Hosts and Domain Registrars Today @ hostacquisitions.co.uk
    UK Web Hosting | UK VPS | UK Dedicated Servers | ADSL/FTTC | Backup/DR | Cloud
    UK Colocation | Reseller Accounts | IPv6 Transit | Secondary MX | DNS | WHMCS Modules

  5. #5
    Join Date
    Feb 2004
    Location
    Louisville, Kentucky
    Posts
    1,083
    Originally posted by othellotech
    ospf - open shortest path first
    othellotech, I think you may misunderstand the purpose, capabilities, and ISP implementation (modern and otherwise) of OSPF. Perhaps you picked up a buzz-word and repeated it, or perhaps you just haven't had the kind of implementation experience one needs to understand the different uses of interior vs exterior route information.

    Below is some explaination, but in short, your statement to the original poster is wrong / misleading based on their question. They want to be told BGP and they probably want to be told tweak your local-preference on transit-learnt routes to influence how much traffic you egress to AboveNet, Level(3), and XO. I'm pretty certain they don't want to be told how to determine that the path from aggregation router A to border router B (or transit provider AboveNet on border router B) costs 300, while the path to border router D (or transit provider Level(3)) costs 450; which is what OSPF does for a single-POP host.

    Again, below is more of my own discussion on this topic.

    OSPF only knows about the interior cost to reach either interior destinations (in most configurations, which are wrong); or the interior cost to reach external next-hops (right). It doesn't know about as-path length, and indeed in the bestpath selection process, as-path length is compared before the OSPF cost to the protocol next-hop is ever examined.

    OSPF also doesn't know about local-preference, which can be set on a per-route basis; as opposed to OSPF costs, which are configured per-interface. You can't say, increase the OSPF cost of all AOL/ATDN routes learnt via AboveNet because they don't have a unique next-hop. While creating a situation in which they did indeed have a unique next-hop would be a creative and novel implementation, I don't think it's very usable, especially on border routers with multiple transit sessions.

    What is OSPF for? In a single POP network, it's main value is to let you tune, in a very minor way, transit egress; and to propogate loopback address and next-hop reachability within your network. In a multi-POP network, generally you set your OSPF costs based on latency; and this helps you choose routes that are somewhat local, as opposed to shipping traffic from one coast to the other before dumping them on your transit providers or peers.

    To be clear, there are no OSPF routes for individual Internet prefixes. The global BGP table contains over 130,000 routes; and if you put 130,000 routes into your OSPF, your routers would probably fail quickly. In any case, you will lose as-path information, communities, and local-preference; which are helping you make your bestpath decision.
    Jeff at Innovative Network Concepts / 212-981-0607 x8579 / AIM: jeffsw6
    Expert IP network consultation and operation at affordable rates
    95th Percentile Explained Rate-Limiting on Cisco IOS switches

  6. #6
    Join Date
    Nov 2002
    Posts
    2,780
    Well said,

    Here is some more recommendation on Routers:

    Foundry Bigiron with a pair of Management 4 Blade
    Riverstone RS with a pair of CM5
    Juniper M5 may work as well, they're on Ebay for 7.5K currently

  7. #7
    Join Date
    Apr 2001
    Location
    Pittsburgh, PA
    Posts
    1,304
    Originally posted by DeathNova
    Here is a quote that I received not too long ago on a Cisco 7200 series router capable of BGP4 over 2xGigE interfaces,
    In a real-world traffic mix, a 7204VXR with NPE-300 could probably route about 200Mbps total. One DOS attack and it would melt into slag (figuratively speaking).

    Go with the Juniper M5; it's exactly designed for this scenario; low-end multihoming. And because it's a Juniper, everything can run at wire speed in almost every scenario.

    Kevin

  8. #8
    Join Date
    Feb 2004
    Location
    Louisville, Kentucky
    Posts
    1,083
    Riverstone's boxes are flow-based and are vulnerable to DoS attacks that exploit their flow setup rate limitations. If you ever plan to have a customer who is sensitive to DoS, I wouldn't go with Riverstone (unless they've substantially improved the number of new flows they can learn / unit time); as it's not just one customer who'll be taken down in such a situation -- it's all your customers.

    Foundry has precisely the same problem. If you get too many random source/dest packets transiting the box, it'll start expiring flows to free up space in its CAM, and in short order that will begin to affect flows created for legitimate traffic, causing packet loss as it can't keep up with the demand for new flows.
    Jeff at Innovative Network Concepts / 212-981-0607 x8579 / AIM: jeffsw6
    Expert IP network consultation and operation at affordable rates
    95th Percentile Explained Rate-Limiting on Cisco IOS switches

  9. #9
    Join Date
    Apr 2001
    Location
    Pittsburgh, PA
    Posts
    1,304
    Originally posted by jsw6
    Foundry has precisely the same problem. If you get too many random source/dest packets transiting the box, it'll start expiring flows to free up space in its CAM, and in short order that will begin to affect flows created for legitimate traffic, causing packet loss as it can't keep up with the demand for new flows.
    Not all Foundry equipment works this way; their product line is confusing, but look for the newest stuff based on the JetCore chipset. You won't find it in the used market very much, but it has much less trouble with a large number of unique flows.

    Kevin

  10. #10
    Join Date
    Nov 2002
    Posts
    2,780
    There;s the Cisco GSR that nobody had mentioned yet, but it would probably cost about the same as a Juniper. You may as well go for a Juniper.

  11. #11
    Join Date
    Apr 2001
    Location
    Pittsburgh, PA
    Posts
    1,304
    Originally posted by Mfjp
    There;s the Cisco GSR that nobody had mentioned yet, but it would probably cost about the same as a Juniper. You may as well go for a Juniper.
    The M5 is a very small Juniper but still has the benefits of Juniper technology. There are also fixed-configuration versions available. The GSR is a huge chassis-based Cisco with quite a bit more complexity to achieve the same benefits. There is really no "cheap" GSR

    Kevin

  12. #12
    Join Date
    Feb 2004
    Location
    Louisville, Kentucky
    Posts
    1,083
    Originally posted by sigma
    Not all Foundry equipment works this way; their product line is confusing, but look for the newest stuff based on the JetCore chipset. You won't find it in the used market very much, but it has much less trouble with a large number of unique flows.
    The JetCore modules are still flow-based. My understanding of the reason the JetCore platform is less vulnerable to this type of DoS is based on the forwarding behavior when a matching CAM entry isn't found on the ingress module.

    IronCore modules treat the local CAM as a cache, and in the event of a "miss" the CAM return a FID that results in the packet being forwarded to the management module, while being retained in the ingress module's shared memory. The management module then performs a FIB lookup, ACL actions, etc. and sends a FID update back to the ingress module. This makes it possible for the management module to become overwhelmed with new flows.

    While JetCore modules have twice as much CAM space as IronCore for similar-speed port-groups, as I understand, the true reason JetCore modules are less vulnerable in this scenario is that they retain packets for which corrosponding FID information cannot be found in the local CAM in their shared memory and request a lookup from the management module by sending only the first 64 bytes of the packet to the management module.

    Is this your understanding as well, or am I confused?
    Jeff at Innovative Network Concepts / 212-981-0607 x8579 / AIM: jeffsw6
    Expert IP network consultation and operation at affordable rates
    95th Percentile Explained Rate-Limiting on Cisco IOS switches

  13. #13
    I have a question does BGP actually offer better quality bandwidth and faster times?

  14. #14
    Join Date
    Feb 2004
    Location
    Louisville, Kentucky
    Posts
    1,083
    Originally posted by cartmanrules
    I have a question does BGP actually offer better quality bandwidth and faster times?
    BGP offers flexibility that, when taken advantage of by knowledgable net.ops folks, can offer better latency and an excellent degree of control over how much traffic egresses to various transit providers. Your choice of transit providers is important, though. Keep in mind that the true reason folks run BGP is to have redundancy and egress traffic control. Only Internap and the new black box vendors such as net*vmg and RouteScience attempt to directly reduce latency to individual Internet destinations in concert with BGP-learnt routing information.
    Jeff at Innovative Network Concepts / 212-981-0607 x8579 / AIM: jeffsw6
    Expert IP network consultation and operation at affordable rates
    95th Percentile Explained Rate-Limiting on Cisco IOS switches

  15. #15
    Join Date
    Apr 2001
    Location
    Pittsburgh, PA
    Posts
    1,304
    We're actually moving to using our Foundry switches for Layer 2 only, where the new gear can easily do wire-speed.

    Kevin

  16. #16
    Join Date
    Dec 2003
    Location
    Canton, Ohio
    Posts
    62
    If you cannot afford the Cisco solution just yet, I recommend running Linux + Zebra BGP. Since this will be a fairly cheap solution you can probably afford to buy two servers to handle your routing. I have never routed 2x GigE, on a Linux machine before. This might work while your starting out with 10/100 bandwidth. Maybe once you get over GigE in bandwidth a Cisco router won't seem so expensive. Just a thought,

    Dustin Wright
    The Karcher Group

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •