Page 1 of 2 12 LastLast
Results 1 to 25 of 38
  1. #1
    Join Date
    Feb 2005
    Location
    Minnesota
    Posts
    973

    Open source routers, performance, stability, features.

    So I was thinking of testing out some open source options before our next major router upgrades are due.

    I was curious if anyone here is utilizing any of the open source routers for production and if so what platform that you are using... likes, dislikes, performance, stability, features, reporting, etc?
    AfterNorth Innovative solutions for tomorrow, today.
    0spam.org AntiSpam for Service Providers
    DotNetInvoice Online Billing Solutions
    Professional Services Since 1996

  2. #2
    Join Date
    Aug 2004
    Location
    Canada
    Posts
    3,785
    Quote Originally Posted by Kiamori View Post
    So I was thinking of testing out some open source options before our next major router upgrades are due.

    I was curious if anyone here is utilizing any of the open source routers for production and if so what platform that you are using... likes, dislikes, performance, stability, features, reporting, etc?
    I'd be curious to hear some production use as well. Vyatta has claims of 11 million pps per CPU core on their vPlane platform in their marketing materials: http://www.vyatta.com/learn/road-to-SDN . However if you read these parts most experiences suggest not being able to do more than 1gbit and 1 million pps on a modern CPU.
    Tony B. - Chief Executive Officer
    Hawk Host Inc. Proudly serving websites since 2004
    Quality Shared and Cloud Hosting
    PHP 5.2.x - PHP 8.1.X Support!

  3. #3
    Join Date
    Apr 2009
    Location
    Romania
    Posts
    473
    For BGP and OSPF I usually go with a modern distribution like Ubuntu and intel network cards for offloading.

    For office filtering pfSense is what I like.

    It all depends on what you are trying to do with those routers.
    @TonyB - I have to disagree here. If every single packet hits the CPU than you are right, but with modern network cards this should not happen.
    PidginHost.com - Managed dedicated servers, cloud servers and software development.

  4. #4
    Join Date
    Oct 2006
    Location
    US/EU/UK
    Posts
    4,886
    We have used Quagga at the time before migrating to Juniper. It is stable and we never had issues. It lacks some advanced features, but it should work for smaller networks.
    HostColor.com Edge Infrastructure - US Dedicated Servers & Europe Dedicated Hostingsince 2000
    In 50 U.S. Edge Data Centers & 80 POPs worldwide
    24/7 Support ★★ Support Tickets - LiveChat - Phone

  5. #5
    Join Date
    Feb 2005
    Location
    Minnesota
    Posts
    973
    Quote Originally Posted by HostColor View Post
    We have used Quagga at the time before migrating to Juniper. It is stable and we never had issues. It lacks some advanced features, but it should work for smaller networks.
    It would need to have better performance/$ than cisco without compromising stability and security. Also needs to support advanced usage reporting like netflow to even be considered. I used a few Juniper routers back in the T3/T1 days and did not really like them.
    AfterNorth Innovative solutions for tomorrow, today.
    0spam.org AntiSpam for Service Providers
    DotNetInvoice Online Billing Solutions
    Professional Services Since 1996

  6. #6
    Join Date
    Oct 2002
    Location
    Vancouver, B.C.
    Posts
    2,699
    Quote Originally Posted by Kiamori View Post
    It would need to have better performance/$ than cisco without compromising stability and security. Also needs to support advanced usage reporting like netflow to even be considered. I used a few Juniper routers back in the T3/T1 days and did not really like them.
    When it comes to routing, no general purpose computer is going to hold a candle to a proper ASICs based solution from vendors like Juniper or Cisco. x86 machines running open source just cannot come anywhere close to the PPS throughput of a proper routing appliance.

    Firewalls are a different story. Except at the very high end, most firewalls are just as if not more susceptible to high PPS than a good x86 build running a BSD or Linux variant.
    ASTUTE INTERNET: Advanced, customized, and scalable solutions with AS54527 Premium Performance and Canadian Optimized Network (Level3, Shaw, CogecoPeer1, GTT/Tinet),
    AS63213 Cost Effective High Performance Network (Cogent, HE, GTT/Tinet)
    Dedicated Hosting, Colo, Bandwidth, and Fiber out of Vancouver, Seattle, LA, Toronto, NYC, and Miami

  7. #7
    Quote Originally Posted by Kiamori View Post
    I was curious if anyone here is utilizing any of the open source routers for production and if so what platform that you are using... likes, dislikes, performance, stability, features, reporting, etc?
    I think you'll find it hard to get people to admit to this. Generally people regard software based routers (e.g. a software stack running on commodity x86 hardware) as a cheap alternative to a 'proper' router from the likes of Cisco/Juniper etc.

    There will definitely be people using them, but I don't think they'll want to admit to it if they're using them in a larger installation.

  8. #8
    Join Date
    Aug 2000
    Location
    Sheffield, South Yorks
    Posts
    3,627
    We ran for many years with Quagga on some IBM 2U servers with a mix of copper and fiber Gig E. They served us very well, where quagga falls down though is policy and IPv6. We're a Juniper shop now.
    Karl Austin :: KDAWS.com
    The Agency Hosting Specialist :: 0800 5429 764
    Partner with us and free-up more time for income generating tasks

  9. #9
    Quote Originally Posted by KDAWebServices View Post
    We ran for many years with Quagga on some IBM 2U servers with a mix of copper and fiber Gig E. They served us very well, where quagga falls down though is policy and IPv6. We're a Juniper shop now.
    I stand corrected... maybe someone will admit to using software based routers in the past!

    Out of interest, was the switch to Juniper driven by requirements for better policy routing/IPv6, or something else? Were your customers aware you were running Quagga routers?

  10. #10
    Join Date
    Sep 2012
    Location
    NYC
    Posts
    44
    I have seen people running product called mikrotik cloud router, for core routing, full BGP with 6 ISP's.
    NGN Telecommunication Infrastructure as a Service over Ultra-Low Latency Network.

  11. #11
    Join Date
    Aug 2000
    Location
    Sheffield, South Yorks
    Posts
    3,627
    Mikrotik aren't without their issues and foibles either though. Especially at the moment with their move to many cores - some stuff isn't multi-threaded yet I seem to recall and given that each CPU core isn't all that powerful it's causing some issues with full BGP tables for example.
    Karl Austin :: KDAWS.com
    The Agency Hosting Specialist :: 0800 5429 764
    Partner with us and free-up more time for income generating tasks

  12. #12
    Join Date
    Oct 2009
    Location
    Canada
    Posts
    485
    It really all depends on the application. In a large network as the border I wouldn't use them (all that really needs to happen there is packet forwarding in most cases).

    Would I incorporate them into a cloud topology, or custom application in a devOPS environment? Definitely, the sole reason is the API and the PPS at that level of the network is usually substantially lower allowing you to automate a lot of things in code. I know juniper is trying to get themselves into that market by coming up with an API on their developer site.

    That being said, if you're routing a single rack with a single gigE uplink, you can get away with it (have two for redundancy though). If you're routing a full datacenter, definitely not. Even the most budget/low cost hosting providers (ie the ovh & burstnet's of the world) still use Cisco or Juniper in their networks when it comes to scaling big.
    █ Pentester & IT Security Consultant

  13. #13
    Join Date
    Feb 2005
    Location
    Minnesota
    Posts
    973
    Quote Originally Posted by justcolo View Post
    I think you'll find it hard to get people to admit to this. Generally people regard software based routers (e.g. a software stack running on commodity x86 hardware) as a cheap alternative to a 'proper' router from the likes of Cisco/Juniper etc.

    There will definitely be people using them, but I don't think they'll want to admit to it if they're using them in a larger installation.
    I was looking for something all around better than cisco... and was not sure what else is even available in the community built realm. With advancements in tech I don't see why a better open source solution is not available or being implemented by the community or larger companies yet. What do the larger .com's use? I would think that some of them have something custom developed.

    If its better I don't see why it would be an issue, I see your point on bottom of the barrel alternatives.
    AfterNorth Innovative solutions for tomorrow, today.
    0spam.org AntiSpam for Service Providers
    DotNetInvoice Online Billing Solutions
    Professional Services Since 1996

  14. #14
    Join Date
    Oct 2002
    Location
    Vancouver, B.C.
    Posts
    2,699
    Quote Originally Posted by Kiamori View Post
    I was looking for something all around better than cisco... and was not sure what else is even available in the community built realm. With advancements in tech I don't see why a better open source solution is not available or being implemented by the community or larger companies yet. What do the larger .com's use? I would think that some of them have something custom developed.
    High performance routers require extensive use of ASICS, which need to be custom fabricated which is a very expensive process. Even FPGA's will not suffice, let alone general purpose processors. It's not just a matter of developing software. Even the larger .com's do not have sufficient scale to manufacture their own ASICS, so they will use the same network equipment vendors as anyone else. The .com's don't necessarily even need the higher capacity routers, it's more telcos and bandwidth carriers that do. They'll just use higher end models like Juniper's T series instead of the MX series that some hosting and data centre providers use, and Cisco's CRS instead of the ASR series. Nobody makes bigger routers than these.

    Juniper JunOS is based on FreeBSD, which is an open source operating system. Cisco uses QNX which is a real-time operating system also part of the Blackberry 10 OS, Tesla's and many other cars' OS, industrial control systems, embedded systems, etc. Some of QNX's code like its network stack, package management system, and other code come from NetBSD.
    Last edited by hhw; 09-27-2013 at 05:36 PM.
    ASTUTE INTERNET: Advanced, customized, and scalable solutions with AS54527 Premium Performance and Canadian Optimized Network (Level3, Shaw, CogecoPeer1, GTT/Tinet),
    AS63213 Cost Effective High Performance Network (Cogent, HE, GTT/Tinet)
    Dedicated Hosting, Colo, Bandwidth, and Fiber out of Vancouver, Seattle, LA, Toronto, NYC, and Miami

  15. #15
    Join Date
    Mar 2010
    Posts
    4,533
    Vyatta comes to mind. I know of 3-4 vps providers that use that platform even if they don't want to admit it.

    I've found it stable and usable so far. It's obviously not going to compare to a true switch and routers but I have heard a few reports of users running tweaks and special hardware to get close performance of real equipment. Special Nic cards, heavy tweaking etc. None of them have really released the full information though. Which is quite understandable if it gives them a competitive edge in certain areas.

    Most of the software solutions should work for a bit. But if you are already in the 10gbit range and such, I don't see any other reasonable choice other than juniper, cisco, brocade, and some of the cheap Chinese brands. You could probably get a full rack of systems and get something that works that way. But the power cost would likely offset the savings.

    I personally have been trying to get a hold of some of the bitcoin mining asic equipment and see if it's even possible to get these software routers working on them. But the 1 year wait time on some of the orders had me change my mind.

    On a side note, I believe SwiftWay has testing some software router type solutions. Hopefully the member here can chime in too.
    Last edited by techjr; 09-27-2013 at 06:17 PM.

  16. #16
    Join Date
    Jul 2008
    Location
    Manhattan, NY Seattle,WA
    Posts
    4,184
    Quote Originally Posted by techjr View Post
    On a side note, I believe SwiftWay has testing some software router type solutions. Hopefully the member here can chime in too.
    Was hoping to hear more about this from him before however his last post about it said they're not releasing information on it I believe. It would be great to have something that functions at a high rate, I plan on giving a few different setups some tries in the future however we're already running some nice Cisco gear for our network it would more be for testing as it's something interesting and hey if you can get something to work for 3-8k vs having to buy a 20k+ router that's always nice savings. However you to make sure it's then able to handle 40+gbps if you can.

    It would be great to hear from those who have a setup like this.
    ⚡️ PUREVOLTAGE.COM ⚡️Custom Dedicated Servers, Colocation, VPS Contact us: sales@purevoltage.com Skype: Mobile.Jake
    AMD EPYC 7443P RYZEN 7950X3D ⚡️ NVME 10G - 100Gbps We do it all!

    New York City ★ Seattle ★ Los Angeles ★ Chicago ★ Dallas

  17. #17
    Join Date
    Feb 2005
    Location
    Minnesota
    Posts
    973
    Quote Originally Posted by hhw View Post
    High performance routers require extensive use of ASICS, which need to be custom fabricated which is a very expensive process. Even FPGA's will not suffice, let alone general purpose processors. It's not just a matter of developing software. Even the larger .com's do not have sufficient scale to manufacture their own ASICS, so they will use the same network equipment vendors as anyone else. The .com's don't necessarily even need the higher capacity routers, it's more telcos and bandwidth carriers that do. They'll just use higher end models like Juniper's T series instead of the MX series that some hosting and data centre providers use, and Cisco's CRS instead of the ASR series. Nobody makes bigger routers than these.

    Juniper JunOS is based on FreeBSD, which is an open source operating system. Cisco uses QNX which is a real-time operating system also part of the Blackberry 10 OS, Tesla's and many other cars' OS, industrial control systems, embedded systems, etc. Some of QNX's code like its network stack, package management system, and other code come from NetBSD.
    Do you know of any good articles about Application Specific Integrated Circuits for enterprise routers... I don't really know much about this and would love to learn more.
    AfterNorth Innovative solutions for tomorrow, today.
    0spam.org AntiSpam for Service Providers
    DotNetInvoice Online Billing Solutions
    Professional Services Since 1996

  18. #18
    Join Date
    Oct 2002
    Location
    Vancouver, B.C.
    Posts
    2,699
    Quote Originally Posted by Kiamori View Post
    Do you know of any good articles about Application Specific Integrated Circuits for enterprise routers... I don't really know much about this and would love to learn more.
    Just found this presentation from a Juniper engineer at NANOG49:

    http://www.nanog.org/meetings/nanog4...ernRouters.pdf

    Funny, I was actually at that NANOG but have no recollection that there was even such a presentation, but maybe that was because it was in the morning
    ASTUTE INTERNET: Advanced, customized, and scalable solutions with AS54527 Premium Performance and Canadian Optimized Network (Level3, Shaw, CogecoPeer1, GTT/Tinet),
    AS63213 Cost Effective High Performance Network (Cogent, HE, GTT/Tinet)
    Dedicated Hosting, Colo, Bandwidth, and Fiber out of Vancouver, Seattle, LA, Toronto, NYC, and Miami

  19. #19
    Join Date
    Apr 2008
    Location
    United States, MI
    Posts
    769
    Quote Originally Posted by ClaudiuPopescu View Post
    For BGP and OSPF I usually go with a modern distribution like Ubuntu and intel network cards for offloading.

    For office filtering pfSense is what I like.

    It all depends on what you are trying to do with those routers.
    @TonyB - I have to disagree here. If every single packet hits the CPU than you are right, but with modern network cards this should not happen.
    You trust your routing to an OS like Ubuntu?

    On purpose, or was that mandated by someone who doesn't understand computers above you in your food chain?
    Steven Crothers
    No BS cloud engineer and Red Hat architect.

  20. #20
    Join Date
    Apr 2009
    Location
    Romania
    Posts
    473
    Quote Originally Posted by Crothers View Post
    You trust your routing to an OS like Ubuntu?

    On purpose, or was that mandated by someone who doesn't understand computers above you in your food chain?
    At the moment I have in production: Ubuntu 12 LTS, CentOS 6 and Vyatta. All with BGP, OSPF and ~1Gbps of traffic. Some of them with 2 routers in LB and others with 4.
    All of them for different projects and it was dictated by customers. Having a recent kernel and stable network drivers is enough most of the times.

    It is all about the people using this OSes/tools.
    PidginHost.com - Managed dedicated servers, cloud servers and software development.

  21. #21
    Join Date
    Apr 2008
    Location
    United States, MI
    Posts
    769
    Quote Originally Posted by ClaudiuPopescu View Post
    At the moment I have in production: Ubuntu 12 LTS, CentOS 6 and Vyatta. All with BGP, OSPF and ~1Gbps of traffic. Some of them with 2 routers in LB and others with 4.
    All of them for different projects and it was dictated by customers. Having a recent kernel and stable network drivers is enough most of the times.

    It is all about the people using this OSes/tools.
    Why the fragmentation in your environment? I can't imagine a customer ever said "I really want you to choose a new Linux operating system for our specific router so you can announce or small subnet on dedicated x86 based hardware".
    Steven Crothers
    No BS cloud engineer and Red Hat architect.

  22. #22
    Join Date
    Oct 2002
    Location
    Vancouver, B.C.
    Posts
    2,699
    Quote Originally Posted by Crothers View Post
    Why the fragmentation in your environment? I can't imagine a customer ever said "I really want you to choose a new Linux operating system for our specific router so you can announce or small subnet on dedicated x86 based hardware".
    Different clients have different needs. In a hosting environment, you don't always get to pick and choose every platform used. We don't all have the luxury of a completely homogenous environment.

    That aside though, there are perfectly valid reasons for using x86 for routing. If you need a full BGP table, but are on a limited budget and don't need much PPS throughput, a Linux based platform can be an excellent fit. And if this is for the client's dedicated use which they access themselves, which is what it sounds like, it's important that they are comfortable working with that environment as well. Not everyone has cut their teeth on IOS or JunOS.
    ASTUTE INTERNET: Advanced, customized, and scalable solutions with AS54527 Premium Performance and Canadian Optimized Network (Level3, Shaw, CogecoPeer1, GTT/Tinet),
    AS63213 Cost Effective High Performance Network (Cogent, HE, GTT/Tinet)
    Dedicated Hosting, Colo, Bandwidth, and Fiber out of Vancouver, Seattle, LA, Toronto, NYC, and Miami

  23. #23
    Join Date
    Feb 2005
    Location
    Minnesota
    Posts
    973
    Quote Originally Posted by hhw View Post
    Just found this presentation from a Juniper engineer at NANOG49:

    http://www.nanog.org/meetings/nanog4...ernRouters.pdf

    Funny, I was actually at that NANOG but have no recollection that there was even such a presentation, but maybe that was because it was in the morning
    Guess I was looking for something a little more in-depth than that.
    AfterNorth Innovative solutions for tomorrow, today.
    0spam.org AntiSpam for Service Providers
    DotNetInvoice Online Billing Solutions
    Professional Services Since 1996

  24. #24
    Join Date
    May 2011
    Location
    New Zealand
    Posts
    21
    Quote Originally Posted by KDAWebServices View Post
    Mikrotik aren't without their issues and foibles either though. Especially at the moment with their move to many cores - some stuff isn't multi-threaded yet I seem to recall and given that each CPU core isn't all that powerful it's causing some issues with full BGP tables for example.
    I run wireless networks using Mikrotik, the new cloud core stuff is great.
    I know that BGP is not multithreaded yet, it'll use %100 of a cpu depending on the route table.

  25. #25
    Join Date
    Jan 2008
    Location
    Chicago, IL
    Posts
    338
    I know that BGP is not multithreaded yet, it'll use %100 of a cpu depending on the route table.
    I think that's addressed in version 6.3. Not sure yet 100%, but I'll post in the CCR thread once my 2 10Gig models arrive with a review from the workbench.
    Ian Halliday
    nDeviX Corporation
    www.ndevix.com
    Web Development | IT Consulting | Hosted Services | Mobile Apps

Page 1 of 2 12 LastLast

Similar Threads

  1. High performance open source web server- kangle
    By appcomq in forum Hosting Software and Control Panels
    Replies: 2
    Last Post: 01-06-2012, 09:58 AM
  2. Replies: 20
    Last Post: 11-25-2010, 12:18 PM
  3. ServerSignature Open Source Solutions & Open Source Consulting
    By serversignature in forum Employment / Job Requests
    Replies: 0
    Last Post: 07-21-2009, 02:02 AM
  4. [URGENT] Need an open source game panel (counterstrike:source) for linux
    By HostVillage Sales in forum Dedicated Server
    Replies: 0
    Last Post: 10-18-2007, 11:06 PM
  5. E-mail Support Tracking - Open Source - Features & Feedback
    By enhance in forum Hosting Software and Control Panels
    Replies: 2
    Last Post: 06-15-2003, 05:28 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •