Results 1 to 25 of 25
  1. #1

    reliable inexpensive 24 port raid card with kernel driver support?

    Hi Everyone,

    I've got one customer who uses a ton of disks, and we had been setting them up with motherboards with 14 onboard sata and then getting an additional 8 port raid card. This has caused some problems as the 8 port card (highpoint 2320) will lose the raid driver whenever the kernel updates, and since this customer uses ksplice, that tends to be at nearly every reboot.

    In any case, for his next server he's considering just going with a 24 port raid card. Ideally we'd be able to boot linux off of drives connected to the raid card, but at a minimum the OS needs to be able to see the drives on the raid card after we update the kernel without having to reinstall the raid driver. It would be a plus if the card was also reliable and inexpensive. Most likely will be using software raid anyway, so the actual raid features / performance are generally irrelevant here, just need a bunch of sata ports.
    IOFLOOD.com -- We Love Servers
    Phoenix, AZ Dedicated Servers in under an hour
    ★ Ryzen 9: 7950x3D ★ Dual E5-2680v4 Xeon ★
    Contact Us: sales@ioflood.com

  2. #2
    Join Date
    Mar 2009
    Posts
    3,700
    may i ask which board do you use to support 14 sata onboard ? thanx

  3. #3
    Quote Originally Posted by ttgt View Post
    may i ask which board do you use to support 14 sata onboard ? thanx
    supermicro has an x8 motherboard that, for $100 extra, has an 8 port LSI onboard module.
    IOFLOOD.com -- We Love Servers
    Phoenix, AZ Dedicated Servers in under an hour
    ★ Ryzen 9: 7950x3D ★ Dual E5-2680v4 Xeon ★
    Contact Us: sales@ioflood.com

  4. #4
    Join Date
    Dec 2003
    Location
    Ogden, UT
    Posts
    476
    For the price you will likely end up paying for that 24 port raid card, just get a 24 drive supermicro case with built in sas expander, from there you can use pretty much any sas card. We prefer the LSI 9260-4i (about $400 with bbu)
    Daniel Pautz - WebNX, Inc. dan >< WebNX.com
    WebNX.com Enterprise Hosting Solutions – Southern California (Premium Equinix Based DC), Northern Utah (Large 120k Sq' WebNX ran) and NYC Based Servers
    High end Dedicated Servers at reasonable prices on a Premium network with 9x providers route optimized with the Noction IRP

  5. #5
    Join Date
    Jan 2003
    Location
    Chicago, IL
    Posts
    6,957
    Quote Originally Posted by WebNX View Post
    For the price you will likely end up paying for that 24 port raid card, just get a 24 drive supermicro case with built in sas expander, from there you can use pretty much any sas card. We prefer the LSI 9260-4i (about $400 with bbu)
    I agree, why get a 24 port card, etc.? Just go with SAS expanders, MUCH easier to deal with.
    Karl Zimmerman - Founder & CEO of Steadfast
    VMware Virtual Data Center Platform

    karl @ steadfast.net - Sales/Support: 312-602-2689
    Cloud Hosting, Managed Dedicated Servers, Chicago Colocation, and New Jersey Colocation

  6. #6
    Quote Originally Posted by KarlZimmer View Post
    I agree, why get a 24 port card, etc.? Just go with SAS expanders, MUCH easier to deal with.
    That's certainly a good point as the X8 board I've been using that has the extra 8 sata ports is actually presented as two 4 port SAS ports, and the onboard SAS ports (LSI I believe) are detected under linux without installing drivers. If I already had a 24 port case without a SAS expander, any recommendation on what kind of product I should buy for the SAS expander?
    IOFLOOD.com -- We Love Servers
    Phoenix, AZ Dedicated Servers in under an hour
    ★ Ryzen 9: 7950x3D ★ Dual E5-2680v4 Xeon ★
    Contact Us: sales@ioflood.com

  7. #7
    Join Date
    Mar 2003
    Location
    California USA
    Posts
    13,681
    Funkywizard,

    Just an idea:
    As an alternative to replacing the card, you could build a kmod rpm package to avoid having to redo the driver every time. The module would be compiled against the kernel ABI. If they are using normal yum upgraded kernels (it sounds like it due to ksplice), then the module will continue to work in the newer kernels. I do this with a variety of odd ball hardware.

    Also I did not look, elrepo may have a kmod package already created.

    EDIT: some info here: http://wiki.centos.org/HowTos/BuildingKernelModules
    Last edited by Steven; 05-04-2012 at 03:26 AM.
    Steven Ciaburri | Industry's Best Server Management - Rack911.com
    Software Auditing - 400+ Vulnerabilities Found - Quote @ https://www.RACK911Labs.com
    Fully Managed Dedicated Servers (Las Vegas, New York City, & Amsterdam) (AS62710)
    FreeBSD & Linux Server Management, Security Auditing, Server Optimization, PCI Compliance

  8. #8
    Quote Originally Posted by Steven View Post
    Funkywizard,

    Just an idea:
    As an alternative to replacing the card, you could build a kmod rpm package to avoid having to redo the driver every time. The module would be compiled against the kernel ABI. If they are using normal yum upgraded kernels (it sounds like it due to ksplice), then the module will continue to work in the newer kernels. I do this with a variety of odd ball hardware.

    Also I did not look, elrepo may have a kmod package already created.

    EDIT: some info here: http://wiki.centos.org/HowTos/BuildingKernelModules
    Thanks for the suggestion, this is definitely worth looking into.
    IOFLOOD.com -- We Love Servers
    Phoenix, AZ Dedicated Servers in under an hour
    ★ Ryzen 9: 7950x3D ★ Dual E5-2680v4 Xeon ★
    Contact Us: sales@ioflood.com

  9. #9
    Join Date
    May 2007
    Location
    Arizona
    Posts
    1,201
    Check this out:
    http://www.siliconmechanics.com/i365...age-server.php

    We have 3 of these at work, and they work awesome! OR:
    http://www.siliconmechanics.com/i364...age-server.php (2.5" disks)

  10. #10
    Join Date
    Sep 2004
    Location
    Chicago, IL.
    Posts
    216

    Expanders are the way to go.

    I would have to agree that expanders are the way to go. One of the biggest benefits for this is that you have one cable that goes from the raid card to the backplane (an IPASS or 8087 connector cable).

    This not only allows for better airflow, but in the event the backplane were to die, there is only one cable vs 24 quite hard to place cables. Also, you lose eliminate the risk of accidentally hooking up the wrong cable to the wrong drive and someday pull the wrong drive causing yourself problems.


    There are several models Supermicro has:
    http://www.supermicro.com/products/nfo/storage.cfm

    If you have any questions about any configs let me know!

    For 3.5" SATA drives we use this configuration a lot.
    http://www.supermicro.com/products/c...6E16-R1200.cfm

    And we use this 4 port card with a BBU:
    http://www.lsi.com/products/storagec...AS9260-4i.aspx
    Curt Royer - Amadi Systems
    curt@amadisystems.com - http://amadisystems.com

    Rackmount Solutions | Storage Arrays | Networking & Accessories

  11. #11
    Quote Originally Posted by CurtWRoyer View Post
    I would have to agree that expanders are the way to go. One of the biggest benefits for this is that you have one cable that goes from the raid card to the backplane (an IPASS or 8087 connector cable).

    This not only allows for better airflow, but in the event the backplane were to die, there is only one cable vs 24 quite hard to place cables. Also, you lose eliminate the risk of accidentally hooking up the wrong cable to the wrong drive and someday pull the wrong drive causing yourself problems.


    There are several models Supermicro has:
    http://www.supermicro.com/products/nfo/storage.cfm

    If you have any questions about any configs let me know!

    For 3.5" SATA drives we use this configuration a lot.
    http://www.supermicro.com/products/c...6E16-R1200.cfm

    And we use this 4 port card with a BBU:
    http://www.lsi.com/products/storagec...AS9260-4i.aspx
    Thanks for your feedback!

    If we've already got a case (without sas expander) and a motherboard (with sas onboard), is there a good option for getting a SAS expander?

    I really like the idea of the simplified cable management, and the ability to use our existing LSI on-motherboard SAS (which has been reliable for us so far), as well as the 24 bay supermicro chassis (which has also been reliable for us), so overall think this sounds like a great solution, especially considering the SAS backplane model is only about $200 more than a similar case without that feature. In fact, this could turn out to be $100 cheaper *and* better than how we had done things before with the 14 port supermicro board + 8 port raid card, as the raid card costs about $300 and the sas expander version of the chassis is only about $200 extra if memory serves on the cost of the non-expander version of the chassis.
    IOFLOOD.com -- We Love Servers
    Phoenix, AZ Dedicated Servers in under an hour
    ★ Ryzen 9: 7950x3D ★ Dual E5-2680v4 Xeon ★
    Contact Us: sales@ioflood.com

  12. #12
    First let me say that I have been using LSI SAS 1068e, Areca 1680 and LSI SAS 2008 controllers with HP SAS Expanders (PMC sierra basde IIRC) and LSI expanders for a few years now in storage servers with SATA drives. Generally, it works pretty well. I have however seen a few dropouts over the years with SATA drives.

    My big rules of thumb at this point are:
    1. The controller, SAS expander, and all drives should be SAS 2/ SATA III compatible.
    2. Research your configuration beforehand. Read http://gdamore.blogspot.com/2010/08/...reat-idea.html and other notes on mixing SATA and SAS expanders.
    3. At 24+ disks and especially if you use SSDs, you may want to at least find a dual-linking expander <-> controller solution.
    4. Expanders make a lot of sense to save money when you are using hardware RAID 5, 50, 6, 60 because most 24 port cards use onboard expanders and companies charge a ton for them.

    If you have the space, and want to save a few dollars, LSI 9211-8i (+ onboard) or IBM ServeRAID M1015 and flash to IT mode. Low-profile and inexpensive.
    My site dedicated to server and workstation hardware: http://www.servethehome.com

  13. #13
    Join Date
    Sep 2004
    Location
    Chicago, IL.
    Posts
    216
    No problem, I have upgraded the backplane in existing systems for our customers but this doesn't always fit properly. We have changed this on a 24 drive system and it worked fine.

    Yes, the cost saving is also great too. I actually suggest a lot people going with the 826E16 chassis over the 825TQ chassis because it barely costs more to get the expander and 4 port, vs the 8 port system and 8 port card. Then you have 4 more slots for drives

    It also is cheaper to diagnose issues as a $350 raid card is much better than paying $1200 to find out it was not the raid card that was the problem.

    I have actually never tried using the on-board SAS (with 8087 ports) But I imagine it would work. You would probably need an additional key for RAID5 and RAID6 is not supported on the boards we use.
    Curt Royer - Amadi Systems
    curt@amadisystems.com - http://amadisystems.com

    Rackmount Solutions | Storage Arrays | Networking & Accessories

  14. #14
    Join Date
    Jun 2002
    Location
    Waco, TX
    Posts
    5,623
    Quote Originally Posted by funkywizard View Post
    That's certainly a good point as the X8 board I've been using that has the extra 8 sata ports is actually presented as two 4 port SAS ports, and the onboard SAS ports (LSI I believe) are detected under linux without installing drivers. If I already had a 24 port case without a SAS expander, any recommendation on what kind of product I should buy for the SAS expander?
    that onboard is an LSI2008 I know as I use this one sometimes as well

    It works really well except on rebuilds of an array if you have one, why not just get another LSI2008 card?
    http://www.serversdirect.com/product.asp?pf_id=DC2119
    or lsi branded

    http://www.serversdirect.com/product.asp?pf_id=DC7123
    Last edited by (Stephen); 05-04-2012 at 07:38 PM.

  15. #15
    Join Date
    Jun 2002
    Location
    Waco, TX
    Posts
    5,623
    Quote Originally Posted by CurtWRoyer View Post

    I have actually never tried using the on-board SAS (with 8087 ports) But I imagine it would work. You would probably need an additional key for RAID5 and RAID6 is not supported on the boards we use.
    I'd never even try to use RAID5 or 6 on the onboard cards, they really aren't made for it or they would come with it, without needing to buy a 'key'

  16. #16
    Join Date
    Sep 2004
    Location
    Chicago, IL.
    Posts
    216
    Yeah, I agree they aren't made for it. We try not to use the onboard SAS for a reason And we use that card because most of our customers end up wanting the BBU for write cache and wish they had purchased a card that works with a BBU
    Curt Royer - Amadi Systems
    curt@amadisystems.com - http://amadisystems.com

    Rackmount Solutions | Storage Arrays | Networking & Accessories

  17. #17
    Onboard SAS (LSI) is basically just dedicated PCIe 2.0/ 3.0 lanes to a standard controller. More or less, same as the card version just the motherboard manufacturer places the components on the motherboard and does not need to have a PCIe connector for it.

    One big thing to look out for is that RAID 5/6 performance with no onboard cache (backed by capacitor + NAND or battery) really suffers because you are basically writing directly to the drives rather than having a buffer.

    Been working on compiling a resource for LSI controllers:
    LSI SAS 2008 Controllers
    LSI SAS 2108 Controllers
    LSI SAS 2208 Controllers

    Will have the newer LSI SAS 2308 controller listing up next week (finishing up a 2U roundup then a few ASUS reviews this weekend.) Oftentimes purchasing non-LSI branded cards is less expensive which is the driver behind putting all of the like devices on one page.
    My site dedicated to server and workstation hardware: http://www.servethehome.com

  18. #18
    I'm really just looking at software raid in general as I consider it more reliable, and definitely more flexible. We'll be storing large files so a large stripe size (2MB or more) is a must for good performance, which I've never seen a h/w raid card support that. The LSI onboard has been reliable at not dropping the drives / disappearing the drives, unlike the highpoint 2320, where it's pretty common to have one or two drives randomly disappear requiring a reboot to fix, or, in the case of a kernel upgrade, all the drives disappear pending a driver reinstall / reboot.

    So really just looking for a bunch of reliable ports, moreso than a "raid card" per-se. Thanks for all the feedback so far, very very helpful.
    IOFLOOD.com -- We Love Servers
    Phoenix, AZ Dedicated Servers in under an hour
    ★ Ryzen 9: 7950x3D ★ Dual E5-2680v4 Xeon ★
    Contact Us: sales@ioflood.com

  19. #19
    Join Date
    Dec 2003
    Location
    Ogden, UT
    Posts
    476
    In that case keep your current onboard 8 port and add one of the LSI 16 port JBOD cards, I think they are in the 3-400 range and work very well (drivers for everything out of box). We have a number of clients using them for ZFS setups.
    Daniel Pautz - WebNX, Inc. dan >< WebNX.com
    WebNX.com Enterprise Hosting Solutions – Southern California (Premium Equinix Based DC), Northern Utah (Large 120k Sq' WebNX ran) and NYC Based Servers
    High end Dedicated Servers at reasonable prices on a Premium network with 9x providers route optimized with the Noction IRP

  20. #20
    Join Date
    May 2004
    Location
    Atlanta, GA
    Posts
    3,872
    Quote Originally Posted by funkywizard View Post
    Thanks for your feedback!

    If we've already got a case (without sas expander) and a motherboard (with sas onboard), is there a good option for getting a SAS expander?

    I really like the idea of the simplified cable management, and the ability to use our existing LSI on-motherboard SAS (which has been reliable for us so far), as well as the 24 bay supermicro chassis (which has also been reliable for us), so overall think this sounds like a great solution, especially considering the SAS backplane model is only about $200 more than a similar case without that feature. In fact, this could turn out to be $100 cheaper *and* better than how we had done things before with the 14 port supermicro board + 8 port raid card, as the raid card costs about $300 and the sas expander version of the chassis is only about $200 extra if memory serves on the cost of the non-expander version of the chassis.
    well, you can always to replace the backplane w/o SAS expander in SC846A/SC846TQ with the backplane w/SAS expander (BPN-SAS2-846EL1 6G/s), but backplane alone will cost you over $500. with the new backplane, then you can just connect it to the on-board LSI 2008 controller you already have via one single SFF-8087 to SFF-8087 cable.

    since 2x 8-port LSI 2008 non-RAID host adapters or single 16-port host adapter would still cost you $350-$400, it might be worth to look into upgrading the backplane in your SC846 chassis to BPN-SAS2-846EL1. true, it may cost you $100-$150 more, but you only need to deal with one single cable than 24 single-port SATA connectors. if you could manage to resell the old backplane on eBay or Amazon, chance is very good that you end up spending much less money than buying more host adapter(s).
    C.W. LEE, Apaq Digital Systems
    http://www.apaqdigital.com
    sales@apaqdigital.com

  21. #21
    Quote Originally Posted by cwl@apaqdigital View Post
    well, you can always to replace the backplane w/o SAS expander in SC846A/SC846TQ with the backplane w/SAS expander (BPN-SAS2-846EL1 6G/s), but backplane alone will cost you over $500. with the new backplane, then you can just connect it to the on-board LSI 2008 controller you already have via one single SFF-8087 to SFF-8087 cable.

    since 2x 8-port LSI 2008 non-RAID host adapters or single 16-port host adapter would still cost you $350-$400, it might be worth to look into upgrading the backplane in your SC846 chassis to BPN-SAS2-846EL1. true, it may cost you $100-$150 more, but you only need to deal with one single cable than 24 single-port SATA connectors. if you could manage to resell the old backplane on eBay or Amazon, chance is very good that you end up spending much less money than buying more host adapter(s).
    Thanks for the suggestion. It certainly seems like the cleanest solution, while still being cheaper than buying a 24 port raid card.
    IOFLOOD.com -- We Love Servers
    Phoenix, AZ Dedicated Servers in under an hour
    ★ Ryzen 9: 7950x3D ★ Dual E5-2680v4 Xeon ★
    Contact Us: sales@ioflood.com

  22. #22
    Join Date
    Mar 2003
    Location
    California USA
    Posts
    13,681
    Quote Originally Posted by funkywizard View Post
    Thanks for the suggestion. It certainly seems like the cleanest solution, while still being cheaper than buying a 24 port raid card.
    funkywizard,

    A customer actually uses this exact setup in a dozen or so servers. It works out great.
    Steven Ciaburri | Industry's Best Server Management - Rack911.com
    Software Auditing - 400+ Vulnerabilities Found - Quote @ https://www.RACK911Labs.com
    Fully Managed Dedicated Servers (Las Vegas, New York City, & Amsterdam) (AS62710)
    FreeBSD & Linux Server Management, Security Auditing, Server Optimization, PCI Compliance

  23. #23
    Quote Originally Posted by Steven View Post
    funkywizard,

    A customer actually uses this exact setup in a dozen or so servers. It works out great.
    Thanks for letting me know the real world results there, that's always the biggest thing lacking from just looking at a spec sheet.
    IOFLOOD.com -- We Love Servers
    Phoenix, AZ Dedicated Servers in under an hour
    ★ Ryzen 9: 7950x3D ★ Dual E5-2680v4 Xeon ★
    Contact Us: sales@ioflood.com

  24. #24
    Join Date
    Mar 2003
    Location
    California USA
    Posts
    13,681
    Quote Originally Posted by funkywizard View Post
    Thanks for letting me know the real world results there, that's always the biggest thing lacking from just looking at a spec sheet.
    He uses it for his file servers, in mdadm raid. We push a gigabit out of each box pretty much 24/7.
    Steven Ciaburri | Industry's Best Server Management - Rack911.com
    Software Auditing - 400+ Vulnerabilities Found - Quote @ https://www.RACK911Labs.com
    Fully Managed Dedicated Servers (Las Vegas, New York City, & Amsterdam) (AS62710)
    FreeBSD & Linux Server Management, Security Auditing, Server Optimization, PCI Compliance

  25. #25
    Join Date
    Sep 2006
    Location
    The Not So Deep South
    Posts
    931
    http://www.newegg.com/Product/Produc...ard-_-16401184

    This HP SAS Expander supports the LSI 2008 chip and also supports bonding so you can get 12G to the motherboard and keep your current case. The card is very plentiful on ebay, just avoid the yellow printed PCB's. I know there used to be a HP VAR the hung around on HardForum that resold the expander updated with the latest firmware so an HP server/raid card was not needed.

    Personally I've used Chenbro in the past with excellent results spanning multi chassis, and I know their new 6G Expanders also support the LSI 2008
    Last edited by mitgib; 05-05-2012 at 04:38 PM.
    Hostigation.com - High Resource Hosting
    WHM/cPanel Servers for Hosting and Dedicated Needs
    SolusVM VPS Hosting - Big Features, Small Prices
    Like us on Facebook or follow @hostigation on Twitter

Similar Threads

  1. 3Ware 9550SX-8LP 8 Port Raid Card Low Profile
    By GGS - Kaumil in forum Web Hosting Hardware
    Replies: 0
    Last Post: 01-15-2012, 11:20 AM
  2. How to install Raid Card Driver While installing Centos?
    By onimusha333 in forum Dedicated Server
    Replies: 2
    Last Post: 10-31-2011, 12:52 AM
  3. Spec me a 4 port SATA RAID card?
    By uksysadmin in forum Colocation, Data Centers, IP Space and Networks
    Replies: 26
    Last Post: 03-03-2011, 12:22 AM
  4. 8 port PCI-X raid card..
    By Dougy in forum Colocation, Data Centers, IP Space and Networks
    Replies: 8
    Last Post: 04-03-2010, 07:25 PM
  5. RAID card SATA 2 port - Not software RAID
    By stuartornum in forum Dedicated Server
    Replies: 13
    Last Post: 12-03-2007, 10:00 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •