Results 1 to 21 of 21
  1. #1
    Join Date
    Apr 2011
    Location
    USA - MI
    Posts
    142

    Question Help to build SAN/NAS

    Hello,
    I need help to build a SAN / NAS with good value that will connect to a 1U Dual Node where I create some own VPS to hosting.

    I need help choosing the best components to complete what I have:

    SAN/NAS:
    - Supermicro Case (12/16/24 HDD i need choise yet)
    - Supermicro X8STi
    - Intel Xeon 3430
    - 12GB ECC Kingston DDR3 1333
    - 12/16/24 hdds
    - Raid Controler 6Gbps? I need any additional cable?
    - Fiber card?
    - CentOS
    - File System?
    - Software?

    About drives, i need 12/16 or 24 harddrives. I have heard very good at 6Gbps SATA RAID (10?) is it true? What are the HDDs with the best value? WD Black?
    The better choise is create multiple RAID (10?) arrays to have better I/Os?
    What better software to SAN? OpenFiller?

    Hypervisor:

    - Supermicro SC808T
    - 2xSupermicro X8DTT-IBQF
    - 4xCPU What better price/quality? Xeon 56xx?
    - 48GB ECC Kingston DDR3 1333
    - 4x HDD to OS at RAID1 (2 per server)
    - CentOS
    - Virtualization software?
    - This boards have Mellanox ConnectX-2 QDR Infiniband 40Gbps Controller w/ QSFP connector integrated, so no need fiber controler right? I need anything more?

    Fiber Switch?

    About virtualization software... Maybe XEN? How to configure XEN to connect to SAN and manage this?

    Thanks,

  2. #2
    Join Date
    Jul 2008
    Location
    New Zealand
    Posts
    1,208
    /Subscribed - Will be interesting/useful for me as well.

    Any reason your wanting 2 x RAID 1's per hypervisor? I would think 1 would be enough.
    For the CPU's, maybe look into the new Sandy Bridge CPU's.

  3. #3
    Join Date
    Feb 2005
    Location
    Toronto, Canada
    Posts
    5,158
    Hello,

    Are you looking to purchase this hardware and colocate it?

  4. #4
    Join Date
    Apr 2011
    Location
    USA - MI
    Posts
    142
    @bhavicp to RAID1? New Sandy Brige CPUs is skt 1155 correct? I cant fit this in my boards..
    @THAMAN yes, i have own rack

    Thanks,

  5. #5
    Join Date
    Nov 2009
    Location
    Cincinnati
    Posts
    1,583
    Corey at X10 setup an OpenFiler machine for their free hosting and has has great luck with it. They refer to it as their "getto san"
    'Ripcord'ing is the only way!

  6. X8STI is a single socket 1366 board which won't take X3430 socket 1156 CPU. the new X9SCI-LN4F socket 1155 w/Xeon E3 sandy bridge processor (perhaps E3-1220) will be a good choice for a SAN box. it also comes with four iSCSI capable 82574L GB NIC ports on board already, and 3x PCI-E slots for multiple RAID and fibre cards if you need them.

    all RAID edition or Enterprise SATA drives from WD or Seagate are limited to SATA-II (3G/s). thus, SATA-III 6G/s RAID cards are kinda pointless at this time, and often they are not supported by production-grade linux kernels currently.

  7. #7
    Join Date
    Apr 2011
    Location
    USA - MI
    Posts
    142
    Quote Originally Posted by [email protected] View Post
    X8STI is a single socket 1366 board which won't take X3430 socket 1156 CPU. the new X9SCI-LN4F socket 1155 w/Xeon E3 sandy bridge processor (perhaps E3-1220) will be a good choice for a SAN box. it also comes with four iSCSI capable 82574L GB NIC ports on board already, and 3x PCI-E slots for multiple RAID and fibre cards if you need them.

    all RAID edition or Enterprise SATA drives from WD or Seagate are limited to SATA-II (3G/s). thus, SATA-III 6G/s RAID cards are kinda pointless at this time, and often they are not supported by production-grade linux kernels currently.
    Right, my bad. I have the board and I7 now. Any suggestions to other components?

    Thanks,

  8. #8
    Join Date
    Feb 2002
    Location
    New York, NY
    Posts
    4,612
    Quote Originally Posted by pleiades View Post
    - This boards have Mellanox ConnectX-2 QDR Infiniband 40Gbps Controller w/ QSFP connector integrated, so no need fiber controler right?
    If you're using Infiniband for your storage network, then you don't need Fibre Channel.

    Quote Originally Posted by pleiades View Post
    Fiber Switch?
    You could connect the servers directly, or you could get a switch like these:

    http://www.mellanox.com/content/page...enu_section=49
    Scott Burns, President
    BQ Internet Corporation
    Remote Rsync and FTP backup solutions
    *** http://www.bqbackup.com/ ***

  9. #9
    Join Date
    Apr 2003
    Location
    Los Angeles, CA
    Posts
    800
    Quote Originally Posted by Visbits View Post
    They refer to it as their "getto san"
    Our ghetto SAN consists of a OpenSolaris box running ZFS, with 16 x 2 TB SATA drives connected to two LSI controllers (8 drives each), in RAIDZ2 configuration, plus a 10k rpm ZIL drive (for NFS). It's in a 4U Supermicro case, with a single 4-core CPU and 24 GB of RAM. It serves NFS and iSCSI to ~160 compute nodes over link-aggregated Gbit Ethernet. We actually have two of these running for almost a year. So far so good.

    I don't have IOPS numbers, but reading easily saturates the network interfaces even with small files. Local sequential write performance is ~950 MB/sec (50 GB test file).

    I do like the data and meta data checksumming, transparent compression, data deduplication and file system shapshots (all optional). The CPU isn't being too taxes by all of this.
    Pings <1 ms, Unlimited Transfer, Lowest Price: http://localhost/

  10. #10
    Quote Originally Posted by luki View Post
    Our ghetto SAN consists of a OpenSolaris box running ZFS, with 16 x 2 TB SATA drives connected to two LSI controllers (8 drives each), in RAIDZ2 configuration, plus a 10k rpm ZIL drive (for NFS). It's in a 4U Supermicro case, with a single 4-core CPU and 24 GB of RAM. It serves NFS and iSCSI to ~160 compute nodes over link-aggregated Gbit Ethernet. We actually have two of these running for almost a year. So far so good.

    I don't have IOPS numbers, but reading easily saturates the network interfaces even with small files. Local sequential write performance is ~950 MB/sec (50 GB test file).

    I do like the data and meta data checksumming, transparent compression, data deduplication and file system shapshots (all optional). The CPU isn't being too taxes by all of this.
    Not all that ghetto. I like the idea of your setup.

  11. #11
    Join Date
    Nov 2009
    Location
    Cincinnati
    Posts
    1,583
    Yeah that's a pretty nice setup :-)

    If I knew enough about Solaris to be dangerous I'd try it.
    'Ripcord'ing is the only way!

  12. #12
    Join Date
    Jun 2010
    Location
    Los Angeles
    Posts
    151
    I built this: http://i.imgur.com/JuzQo.jpg

    * Based on Backblaze Storage Pod with many improvements
    * 4U
    * 56+2 (system) disk bays (SAS/SATA 6-gbit)
    * Redundant power
    * 3 RAID controllers w/ 4GB cache each
    * Extended ATX motherboard support

  13. #13
    Join Date
    Oct 2001
    Posts
    1,315
    Quote Originally Posted by luki View Post
    Our ghetto SAN consists of a OpenSolaris box running ZFS, with 16 x 2 TB SATA drives connected to two LSI controllers (8 drives each), in RAIDZ2 configuration, plus a 10k rpm ZIL drive (for NFS). It's in a 4U Supermicro case, with a single 4-core CPU and 24 GB of RAM. It serves NFS and iSCSI to ~160 compute nodes over link-aggregated Gbit Ethernet. We actually have two of these running for almost a year. So far so good.
    ...
    Can I ask what you use for redundancy/backup/failover ?

    All the best,
    Avi Brender
    Reliable Web Hosting by Elite Hosts, Inc
    CPANEL Reseller Hosting - Fantastico - Rvskins - ClientExec

  14. #14
    Join Date
    Apr 2011
    Location
    USA - MI
    Posts
    142
    Hello,
    Any more helps please?

    I need advice to:

    -Best SATA harddrives (Wd Blacks 6Gbps?)
    -HDDs to OS in NAS/SAN and Hypervisor is a important thing?
    -Raid Cards (to 16 hdds maybe)
    -The better choise is create multiple RAID (10?) arrays to have better I/Os?
    -Fiber Network card to SAN/NAS and cable to connect to hypervisors (two ports to eliminate switch needs for now)
    -SAN/NAS filesystem and software
    -Hypervisor software

    Thanks,

  15. #15
    luki - are you using Nexenta?

    Also do you mean 160 hypervisors or instances? Sounds impressive.

  16. #16
    Join Date
    Apr 2003
    Location
    Los Angeles, CA
    Posts
    800
    Quote Originally Posted by nzych View Post
    luki - are you using Nexenta?
    Also do you mean 160 hypervisors or instances? Sounds impressive.
    No, just plain OpenSolaris 10u9. This setup is not for web hosting. It's a file server for a computational cluster with 720 CPU cores (and some GPUs) on 160 physical nodes running Rocks. We do scientific number crunching there... the server has to be able to keep up when someone starts 720 concurrent jobs all wanting to load some data to start processing.

    Quote Originally Posted by MaB View Post
    Can I ask what you use for redundancy/backup/failover ?
    We back up to another (old) 16 x 2 TB disk in RAID5 on another machine (Infortrend RAID connected via FC). This was our primary storage server but we retired it as it couldn't keep up as we added more nodes. The ZFS box easily outperforms it, especially with random I/O.

    The storage servers have two power supplies connected to two UPS' and IPMI management. But failover is manual to that second backup box. That's OK for us.

    Solaris is different from Linux, no doubt. But I find management actually easier. The uncertain future of ZFS and OpenSolaris is something one has to wonder about...
    Pings <1 ms, Unlimited Transfer, Lowest Price: http://localhost/

  17. #17
    Join Date
    Apr 2002
    Location
    Auckland - New Zealand
    Posts
    1,572
    Had better experience with Nexenta opposed to Open solaris, it had the driver support that we wanted built in. Great OS for a filer.

  18. #18
    Join Date
    Apr 2003
    Location
    Los Angeles, CA
    Posts
    800
    Thanks for the info, Steven. I'll check it out.
    Pings <1 ms, Unlimited Transfer, Lowest Price: http://localhost/

  19. #19
    Join Date
    Apr 2011
    Location
    USA - MI
    Posts
    142
    Hello,
    I will use VMWare ESXi in Hypervisor. ITs a good choise?
    I need choise better (speed) SATA drivers (6Gbps?), raid card, fiber card and software to NAS/SAN..

    Any help?

    Thanks,

  20. #20
    Join Date
    Aug 2004
    Location
    Dallas, TX
    Posts
    3,507
    Quote Originally Posted by InTheEP View Post
    I built this: http://i.imgur.com/JuzQo.jpg

    * Based on Backblaze Storage Pod with many improvements
    * 4U
    * 56+2 (system) disk bays (SAS/SATA 6-gbit)
    * Redundant power
    * 3 RAID controllers w/ 4GB cache each
    * Extended ATX motherboard support
    What did you improve?
    Dallas Colocation by Incero, 8 years and counting!
    e: sales(at)incero(dot)com 855.217.COLO (2656)
    Colocation & Enterprise Servers, SATA/SAS/SSD, secure IPMI/KVM remote control, 100% U.S.A. Based Staff
    SSAE 16, SAS70, Redundant Power & Network, Fully Diverse Fiber

  21. #21
    Join Date
    Apr 2011
    Location
    USA - MI
    Posts
    142
    I found this fully redundant supermicro system, what they find to SAN? Apparently the only doubt I have is if you could install a battery to the RAID controller... And LSI 2008 is a good controller?

    http://www.supermicro.com/products/s...6036ST-6LR.cfm

    Thanks,

Similar Threads

  1. do you use any software to build a nas ?
    By ttgt in forum Hosting Software and Control Panels
    Replies: 1
    Last Post: 06-05-2009, 09:30 PM
  2. NAS vs SAN
    By ti_nhatrang in forum Hosting Security and Technology
    Replies: 4
    Last Post: 07-28-2007, 05:21 PM
  3. NAS or SAN
    By marutigurav in forum Web Hosting Lounge
    Replies: 1
    Last Post: 02-13-2007, 02:07 PM
  4. SAN vs BigIP vs NAS
    By myusername in forum Hosting Security and Technology
    Replies: 0
    Last Post: 03-10-2006, 08:04 PM
  5. SAN NAS Solution
    By natxanadu in forum Web Hosting
    Replies: 0
    Last Post: 05-25-2004, 11:23 AM

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •