Results 1 to 24 of 24
  1. #1
    Join Date
    Oct 2004
    Posts
    632

    ZFS backup storage server

    We are looking for good value central backup server for all of our cpanel servers, I googled and found that RAID-Z2 (similar to RAID-6) seems a good option and we want to setup it on software based without any raid card

    my question are :

    * to how many HDs that we will hit performance bottleneck? so we could evaluate cost/performance

    * and any budget and smart method to boost performance up?

    thanks!

  2. #2
    Join Date
    Feb 2004
    Location
    UK
    Posts
    1,431
    Hi

    Have you looked at this ?

    http://nexentastor.org/ <- will do 16TB for free

    so 4 x 4tb
    or
    8 x 2tb
    16 x 1tb

    From experience the more spindles the better the performance, but the higher the running costs (ie more disks)

    Thanks

  3. #3
    Join Date
    May 2007
    Location
    Bhopal - India
    Posts
    332
    8 x 2TB should be sufficient for a medium size deployment, enable compression to save storage and add a ZIL SSD for better write performance.
    LoopByte
    India Based Dedicated Servers, VPS & Hosting Services

  4. #4
    Join Date
    Dec 2011
    Location
    Tulsa, OK
    Posts
    359
    +1 for nexenta.
    OCOSA Communications | Since 2003
    http://www.ocosa.com
    Domains, Hosting, Business Solutions, and Connectivity

  5. #5
    Join Date
    Feb 2002
    Location
    New York, NY
    Posts
    4,618
    Make sure that you test it well before using ZFS. We used ZFS (with RAID-Z2) on some backup servers a few years ago, and had a terrible experience, on both Solaris and FreeBSD.

    Assuming that you have lots of RAM, and that they have fixed the stability issues, you should get decent performance in the beginning. The problem is that it gets extremely fragmented over time. Within months, the servers were becoming barely usable, which made it all the more difficult to get rid of ZFS.

    As far as I know, there's still no online defragmentation feature, which is what is needed to make ZFS usable as a central backup server. It might work well enough for you, but keep a close eye on it.
    Scott Burns, President
    BQ Internet Corporation
    Remote Rsync and FTP backup solutions
    *** http://www.bqbackup.com/ ***

  6. #6
    Join Date
    Mar 2009
    Location
    NL
    Posts
    594
    Quote Originally Posted by bqinternet View Post
    Make sure that you test it well before using ZFS. We used ZFS (with RAID-Z2) on some backup servers a few years ago, and had a terrible experience, on both Solaris and FreeBSD.

    Assuming that you have lots of RAM, and that they have fixed the stability issues, you should get decent performance in the beginning. The problem is that it gets extremely fragmented over time. Within months, the servers were becoming barely usable, which made it all the more difficult to get rid of ZFS.

    As far as I know, there's still no online defragmentation feature, which is what is needed to make ZFS usable as a central backup server. It might work well enough for you, but keep a close eye on it.
    There are no stability issues. We have about 10 servers running Opensolaris / Nexenta Core with an uptime > 1 year (one > 2 years). Zero crashes or issues on all of them. Different clients using them and all kinds of disks (mostly nearline SAS though). All Supermicro servers.
    YISP - High Bandwidth dedicated servers and colocation in YISP-AS(Amsterdam)!

    Website: http://www.yisp.nl
    Contact: info "(AT)" yisp.nl

  7. #7
    Join Date
    May 2007
    Location
    Bhopal - India
    Posts
    332
    We also have few Nexenta/OpenIndiana boxes running pretty well.
    LoopByte
    India Based Dedicated Servers, VPS & Hosting Services

  8. #8
    Join Date
    Jan 2012
    Location
    UK
    Posts
    236
    word of warning, nexenta license says you cannot use it in commercial or business use.

    have no heard anyone getting sued or contacted about it, but you never know.

    http://www.nexentastor.org/projects/...mmunityEdition

    Production use is not allowed with Community edition. It is designed and intended to help people become familiar with the product or for hobbiest use.
    and from EULA

    2.2 If You have acquired a Community Edition license, the total amount of RAW Storage Space is limited as specified on the Site and is subject to change without notice. The Community Edition may ONLY be used for educational, academic and other non-commercial purposes expressly excluding any commercial usage. The Trial Edition licenses may ONLY be used for the sole purposes of evaluating the suitability of the Product for licensing of the Enterprise Edition for a fee. If You have obtained the Product under discounted educational pricing, You are only permitted to use the Product for educational and academic purposes only and expressly excluding any commercial purposes.
    alternative would be :

    http://www.napp-it.org

    on any supported solarwinds based distribution

  9. #9
    Join Date
    Apr 2010
    Posts
    493
    Quote Originally Posted by PeterPP View Post
    We are looking for good value central backup server for all of our cpanel servers, I googled and found that RAID-Z2 (similar to RAID-6) seems a good option and we want to setup it on software based without any raid card

    my question are :

    * to how many HDs that we will hit performance bottleneck? so we could evaluate cost/performance

    * and any budget and smart method to boost performance up?

    thanks!
    ZFS likes ram and likes it even more as you use more and more features.

    As to performance a single 7200 rpm sata drive can take more than a single ge can feed it. Even running a pair of 20gbs infiniband connections full out were not pushing our 48 drive backup heads (144TB in 4ru 192TB if we bought more expensive per GB drives). That said look hard at what your using to back up all backup apps are not created equal some can really stress out the servers and/or the backup heads. We do treat backup drives like tape so no raid the backup system makes redundant copies on multiple backup heads and no random io trashing performance.

  10. #10
    +1 for napp-it.org. The main force behind that is active on our fourms.

    The big thing with ZFS software RAID is that you use compatible hardware and test beforehand. Great solution though. Performance wise for backup no issue. You could also use RAID-Z3 for a larger set of disks if you wanted.
    My site dedicated to server and workstation hardware: http://www.servethehome.com

  11. #11
    Join Date
    Mar 2003
    Location
    chicago
    Posts
    1,781
    all the zfs issues i have seen has been a lack of zfs knowledge not zfs

  12. #12
    Join Date
    Oct 2004
    Posts
    632
    http://www.nexentastor.org/ does looks good! thanks!

  13. #13
    Join Date
    Feb 2004
    Location
    UK
    Posts
    1,431
    Quote Originally Posted by PeterPP View Post
    http://www.nexentastor.org/ does looks good! thanks!
    Yep .. it is, but can get a little expensive if you need more than the 18TB or raw storage.

  14. #14
    Join Date
    Oct 2004
    Posts
    632
    just want to share this nice article
    http://www.solarisinternals.com/wiki...ractices_Guide

    then I have a question after reading, article says

    **For better performance, a mirrored configuration is strongly favored over a RAIDZ configuration particularly for large, uncacheable, random read loads.**

    is it a better than raidZ for backup purpose?

    also, if I go with 2TBx6 with raidz2 for beginning, is it easy to add more disks to raidz2 in the future when needed?

  15. #15
    Join Date
    Oct 2004
    Posts
    632
    and what about
    http://www.freenas.org/
    http://www.nas4free.org/

    should be no 16TB limitations?

  16. #16
    Join Date
    Jun 2003
    Location
    Auckland, New Zealand
    Posts
    30
    Quote Originally Posted by PeterPP View Post
    **For better performance, a mirrored configuration is strongly favored over a RAIDZ configuration particularly for large, uncacheable, random read loads.**

    is it a better than raidZ for backup purpose?
    No because it's a waste of disks, go with RAID-Z2 or Z3 for backup.
    Quote Originally Posted by PeterPP View Post
    also, if I go with 2TBx6 with raidz2 for beginning, is it easy to add more disks to raidz2 in the future when needed?
    First you have to understand how ZFS works. ZFS works with pools and each pool has at least one vdev. Each vdev has a set of disks in it and provides the IOP performance of one disk. So the more vdev's the more performance.

    So can you expand a 6 disk RAID-Z2? Yes. Cheaply? No.

    You cannot expand a vdev but you can add more vdev's to a pool, which will increase IOPs and storage. You can mix and match different vdev sizes but it's not recommended, so another vdev of 6 disks would be required to expand it.

    I just use straight FreeBSD, you could install ZFSGuru if you needed a webUI.

  17. #17
    Join Date
    Oct 2004
    Posts
    632
    can I ask you guys that which system of ZFS I could use to "sell" XFS backup space easily with add/delete/upgrade user function and could set disk/bandwidth limitations on WEB GUI?

    did look at all system mentioned above, none of them could do these

  18. #18
    Spend a little extra time and go with Solaris 11, the license is around $1k /year including support. They run the most current version of ZFS with all the features you would hope for. We run a raid 10 like setup on our system, iSCSI and NFS. Running a Zpool is a walk in the park once you've played with it. The documentation is good, and will walk you over all the setup and configuration, and it's rock solid.

    Code:
    zpool status
      pool: fast
     state: ONLINE
      scan: none requested
    config:
    
            NAME                       STATE     READ WRITE CKSUM
            fast                       ONLINE       0     0     0
              mirror-0                 ONLINE       0     0     0
                c0t50014EE0AE14A803d0  ONLINE       0     0     0
                c0t50014EE6AD813A2Dd0  ONLINE       0     0     0
              mirror-1                 ONLINE       0     0     0
                c0t50014EE6AD8132EFd0  ONLINE       0     0     0
                c0t50014EE6AD8132FAd0  ONLINE       0     0     0
              mirror-2                 ONLINE       0     0     0
                c0t50014EE0036AD948d0  ONLINE       0     0     0
                c0t50014EE058C01ADDd0  ONLINE       0     0     0
              mirror-3                 ONLINE       0     0     0
                c0t50014EE602D5CF31d0  ONLINE       0     0     0
                c0t50014EE602D69542d0  ONLINE       0     0     0
            logs
              mirror-4                 ONLINE       0     0     0
                c0t5001517BB2A9B8E7d0  ONLINE       0     0     0
                c0t5001517BB2A9BA97d0  ONLINE       0     0     0
            cache
              c0t500253855003DC8Cd0    ONLINE       0     0     0
              c0t500253855003DC90d0    ONLINE       0     0     0
    
    errors: No known data errors
    
      pool: rpool
     state: ONLINE
      scan: resilvered 10.1G in 0h1m with 0 errors on Mon Jan  7 16:13:04 2013
    config:
    
            NAME        STATE     READ WRITE CKSUM
            rpool       ONLINE       0     0     0
              mirror-0  ONLINE       0     0     0
                c8d0    ONLINE       0     0     0
                c9d0    ONLINE       0     0     0
    
    errors: No known data errors
    
      pool: slow
     state: ONLINE
      scan: none requested
    config:
    
            NAME                       STATE     READ WRITE CKSUM
            slow                       ONLINE       0     0     0
              mirror-0                 ONLINE       0     0     0
                c0t5000C5004E4C7469d0  ONLINE       0     0     0
                c0t5000C5004E46E899d0  ONLINE       0     0     0
              mirror-1                 ONLINE       0     0     0
                c0t5000C5004E46EF32d0  ONLINE       0     0     0
                c0t5000C5004E482DA6d0  ONLINE       0     0     0
    
    zpool list
    NAME    SIZE  ALLOC   FREE  CAP  DEDUP  HEALTH  ALTROOT
    fast   2.17T  52.7G  2.12T   2%  1.00x  ONLINE  -
    rpool  55.5G  10.7G  44.8G  19%  1.00x  ONLINE  -
    slow   5.44T  1.71T  3.73T  31%  1.00x  ONLINE  -

  19. #19
    Join Date
    Jan 2012
    Location
    UK
    Posts
    236
    We use Sun X4540 machines (48 drives in 4U) for backup servers

    would recommend either going napp-it or solaris 11.

    First because Nexenta is not free for comercial use, second because we found issues on multiple platforms with detection and ejection of dead drives ( dead drive manifested with a normal looking zpool but ALL iscsi target just went dead on the machine till we removed the physical drive from the machine).

  20. #20
    Solaris 11 does make a lot of sense, especially given the newer features available.

    If you wanted to play, here is an OpenIndiana + napp-it installation. We are going to add a bunch more of these ZFS guides.
    My site dedicated to server and workstation hardware: http://www.servethehome.com

  21. #21
    Join Date
    Mar 2002
    Location
    Philadelphia, PA
    Posts
    2,517
    Check out OmniOS, we recently migrated our storage servers to it from Nexenta. It's a light weight, runs very well and they offer commercial support.

    Each of our OmniOS servers has 20TB of space, 1TB of SSD capacity (L2-cache), 196GB memory (L1-cache) and we use a ZeusRAM SSD for the ZIL. These servers are getting hammered all day in production and we've never had any issues with OmniOS or performance.

  22. #22
    Quote Originally Posted by viGeek View Post
    Check out OmniOS, we recently migrated our storage servers to it from Nexenta. It's a light weight, runs very well and they offer commercial support.

    Each of our OmniOS servers has 20TB of space, 1TB of SSD capacity (L2-cache), 196GB memory (L1-cache) and we use a ZeusRAM SSD for the ZIL. These servers are getting hammered all day in production and we've never had any issues with OmniOS or performance.
    And OmniOS is something we will be covering with napp-it integration.

    A bit time and resource limited.
    My site dedicated to server and workstation hardware: http://www.servethehome.com

  23. #23
    Join Date
    Mar 2002
    Location
    Philadelphia, PA
    Posts
    2,517
    Quote Originally Posted by pjkenned View Post
    And OmniOS is something we will be covering with napp-it integration.

    A bit time and resource limited.
    Can't wait, really like what you're doing with the site. I'd love to see some new benchmarks from the S1200 series Atom cpus from you guys.

  24. #24
    Quote Originally Posted by viGeek View Post
    Can't wait, really like what you're doing with the site. I'd love to see some new benchmarks from the S1200 series Atom cpus from you guys.
    Working on that! Thanks for the feedback.

    The other project right now that is taking a TON of time is the colo build-out. Now that the hardware side is done, picking what software to run for the cluster is something I'm less equipped for.

    I will say, I am fairly impressed with the price/ performance of the Dell C6100's. Will be doing more on that.

    Once those are out of the way, we can get some more on the OmniOS + napp-it installation. We did find one nuance to the OmniOS and napp-it installation but are working through it.
    My site dedicated to server and workstation hardware: http://www.servethehome.com

Similar Threads

  1. Storage server for backup
    By host-master in forum VPS Hosting
    Replies: 12
    Last Post: 12-31-2011, 09:27 AM
  2. Opinions of NexentaStor or other ZFS storage for production systems?
    By eger in forum Hosting Software and Control Panels
    Replies: 1
    Last Post: 01-25-2011, 11:33 PM
  3. Replies: 27
    Last Post: 06-20-2010, 03:12 AM
  4. NetDepot's Backup Storage Server
    By IGXHost in forum Managed Hosting and Services
    Replies: 6
    Last Post: 11-26-2009, 04:08 PM
  5. server for online backup storage
    By PostiniWholesaler in forum Dedicated Server
    Replies: 1
    Last Post: 09-02-2005, 02:13 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •