Web Hosting Talk : Web Hosting Main Forums : Colocation and Data Centers : Question regarding network architecture

[tulix]

Question regarding network architecture

We are planning to switch to the new (for us) network architecture.
Right now we are using 3 x 65xx core routers and then 35xx distribution and 29xx for each rack. For high bandwidth usage we are plugging servers directly to 35xx switches. Problem is management and potential bottlenecks, as well as distribution of our 10 Gb/s connections.
We are planning to switch to 2 layer architecture.
Keep 3 x 65xx core routers but for the next layer to use 6513 switches with 10 Gb/s connections in between. And have servers plugged in directly to 6513s. More expensive and mess with longer cables but much easier to manage a mix of high/moderate bandwidth customers.
I am interested in your opinion – what would be pluses and minuses for each solution. Is that OK in your opinion to pay money for 6513s (also you have to have sup720 to use with 10 Gb/s module) for convenience and guarantying that customers will get promised bandwidth at any time, without any bottleneck on the network?





__________________
See almost daily specials at http://www.homecom.com
Social networking? File sharing? MS, Flash, 3G streaming? High bandwidth? Help with architecture? Need a reverse proxy? Need a colo?
10 Gb/s lines? Custom hosting? Access to meet me room? Get in touch with us at info@tulix.com

[FastServ]

If I were you I'd get rid of the 35xx and 29xx and get some refurb Foundry FESX 448's. 48 port 10/100/1000 with 2x10G modules that do line speed. Even better they use very little power and have dual hot swap power supplies.
Run one in each cabinet and upgrade the uplink to 10G as needed based on customer demand. The 10G modules are field upgradable/installable without reloading the switch.
You can get them on the used/refurb market for <$5000 ea. Nothing on the cisco line comes close in terms of performance/value.





__________________Fast Serv Networks LLCAshburn VA (East Coast) and San Diego CA (West Coast) | Dedicated / Colo / XenServer / Streaming MediaManaged high bandwidth solutions on AS29889

[alexjcampbell]

Not sure why you would want to use 6513s for distribution.
I would go for 4948s (checkout this post on cisco-nsp which explains what I think is the best architecture for this: http://puck.nether.net/pipermail/cis...st/053763.html)

[dkitchen]

I wouldn't reccomend anything Foundry, their gear may be good at times but their support is horrible.
Sounds like a reasonable solution, you definitely don't need that middle distro layer unless your network is massive and it'd be better to cut it out.
Things to bear in mind are obviously your 6513's are relatively expensive, if you need two more ports you're going to have to buy a whole new unit. How many ports are we talking here overall? I think it sounds like a great solution for high port density, but if it's relatively low port density with high bandwidth on each you might want to look at another solution.
Have you considered 3750 stacks?? Will give you your 10Gbps uplinks and a 40gbps backplane between units when stacked. May be somewhat more cost effective for your requirements.
Dan





__________________█ Dan Kitchen | Razorblue Ltd
█ ddi: (+44) (0)845 862 1502 | e: dkitchen@razorblue.com
█ UK Intensive Managed Hosting, Clusters and Colocation.
█ HP Servers, Cisco/Juniper Powered Network, RedHat/Microsoft/VMware Certified.

[FastServ]

As fas as I know, C3750's don't have 10Gb options. FESX with 2X10G option far surpasses a 3750 both in throughput and value. Plus, I can't imagine why you would need to rely on vendor support for an L2 edge switch beyond software updates... I've dealt with foundry support on a few occasions for higher end stuff and haven't any complaints at all.





__________________Fast Serv Networks LLCAshburn VA (East Coast) and San Diego CA (West Coast) | Dedicated / Colo / XenServer / Streaming MediaManaged high bandwidth solutions on AS29889

[Jay Suds]

What you have in mind will clearly work, but is also quite expensive and limits you to 300 foot runs to your servers. Without knowing much about your physical topology, that could be a stretch. You are also talking about a serious investment to home run what I assume is going to be several hundred / thousand cat5e terminations back to your 6513 distro switches.
Clearly, one big thing to consider is how many high usage customers / ports do you have? If the majority of your users are using sub 10Mbps, it doesn't make a lot of sense to invest $$$ to be able to guarantee 1Gbps line speed for each port.
In any event, another product you might want to take a look at is the Juniper EX4200 switches. They are stackable and can do 4x1Gbps or 2x10Gbps uplink modules. They are also working on coming out with something chassis based.





__________________Jay Sudowski // Handy Networks LLCDirector of Technical Services
Operating our own data center in downtown Denver | AS30475Dedicated Server Specials | Colocation Specials

[tulix]

with 1u servers - no need 300 feet runs



Quote:



Originally Posted by Jay Suds


What you have in mind will clearly work, but is also quite expensive and limits you to 300 foot runs to your servers. Without knowing much about your physical topology, that could be a stretch. You are also talking about a serious investment to home run what I assume is going to be several hundred / thousand cat5e terminations back to your 6513 distro switches.
Clearly, one big thing to consider is how many high usage customers / ports do you have? If the majority of your users are using sub 10Mbps, it doesn't make a lot of sense to invest $$$ to be able to guarantee 1Gbps line speed for each port.
In any event, another product you might want to take a look at is the Juniper EX4200 switches. They are stackable and can do 4x1Gbps or 2x10Gbps uplink modules. They are also working on coming out with something chassis based.


The secret is that I already have extra 6513 with 3bxl - jut need to get 10 Gb/s module. Regading the rest modules - above 3bxl - 1 8x1Gb/s fiber (some customers want just fiber) the rest 48x100Mb/s modules. Below 3bxl, one slot for the failover (will have later on, I hope), then 10 Gb/s module and the rest 48x1000Mb/s modules (I think 3 or 4 modules). With 32 servers per rack with 100/1000Mb/s mix - for us it is just one row with 12 racks - not that much distance from that unit. Just several units like that will be OK for our data center(s).
Regarding 3750-s. We've looked at that type of switches from Cisco and Force10 - too expensive - per port for us.
Got recommendation about non Cisco and Force10 switches - they are cheaper, but all what we've got is Cisco based - don't know how to mix them, if it is a good idea.
Thank you for all your input.
Tulix.





__________________
See almost daily specials at http://www.homecom.com
Social networking? File sharing? MS, Flash, 3G streaming? High bandwidth? Help with architecture? Need a reverse proxy? Need a colo?
10 Gb/s lines? Custom hosting? Access to meet me room? Get in touch with us at info@tulix.com

[FastServ]

No problems mixing foundry and cisco. We have a 100% cisco core and 50/50 cisco/foundry edge out east, always 100% stable.





__________________Fast Serv Networks LLCAshburn VA (East Coast) and San Diego CA (West Coast) | Dedicated / Colo / XenServer / Streaming MediaManaged high bandwidth solutions on AS29889

[RyanD]

Quote:



Originally Posted by alexjcampbell


Not sure why you would want to use 6513s for distribution.
I would go for 4948s (checkout this post on cisco-nsp which explains what I think is the best architecture for this: http://puck.nether.net/pipermail/cis...st/053763.html)


If you can get your hands on them (only available in short support currently) the 4900M from cisco is a pretty killer switch, 8 x 10G on the base unit and upgradable to 24x10G and also supports twig-gig modules so you can cut down one of those 10G ports to 2 x 1G ports if you needed to. Fairly cost-effective for 24x10G ports as well.





__________________
███ - WireSix - Dedicated Servers, Custom Configurations, Colocation - Celebrating 4 years!

[tulix]

Sorry, I've missed this post - very interesting



Quote:



Originally Posted by alexjcampbell


Not sure why you would want to use 6513s for distribution.
I would go for 4948s (checkout this post on cisco-nsp which explains what I think is the best architecture for this: http://puck.nether.net/pipermail/cis...st/053763.html)


Let me little bit change my question:
Except higher cost and longer cabling, is there any disadvantages of using 6513 in the architecture that I've described?
I see another one - if swicth will go down (which shoudn't with 2 power and failover sups) 300 - 400 servers will get disconnected.
Anything else, that I am missing?





__________________
See almost daily specials at http://www.homecom.com
Social networking? File sharing? MS, Flash, 3G streaming? High bandwidth? Help with architecture? Need a reverse proxy? Need a colo?
10 Gb/s lines? Custom hosting? Access to meet me room? Get in touch with us at info@tulix.com

[RyanD]

Quote:



Originally Posted by tulix


Let me little bit change my question:
Except higher