Results 1 to 2 of 2
  1. #1
    Join Date
    Nov 2005

    Very Large Disk Partitions - Practical Limit

    We're outgrowing our current bulk storage system and I'd like to solicit opinions.

    With 2 TB disks and a 16 disk array, it's possible to have a single 28 TB volume (after deducting RAID5 parity overhead and a hot-spare disk). I've seen arrays from Aberdeen with 48 and 96 disks, for nearly 200 TB. Windows supports up to 256 TB per volume when 64K cluster sizes are used.

    Our backup system uses a ton of storage space, and it would be far more convenient, and more efficient from a utilization standpoint, to access that space as a single volume. Breaking it up into smaller chunks, such as 2 TB each, means we have to make a "best guess" on balancing actual need.

    For example, if we assign 25 servers to each 2 TB volume for backup storage purposes, some volumes might only see 800 GB of consumption (remaining 1.2 TB allocated but not used) while other volumes might get 1.6 TB used (remaining 400 GB allocated but not used). Key concept: wasted space, because we have to over-estimate need to assure adequate headroom.

    From the opposite viewpoint, if we had a sudden increase in need that exceeded the available space allocated to that volume, we'd have to move that server to a different volume. Key concept: increased admin workload to monitor and re-balance distribution as needed.

    Now if we used one giant volume, there would be no guesswork, no "allocating more than we think is needed" for a bunch of small volumes. All servers share one huge common pot.

    But there has to be a practical limit from a system-overhead standpoint. Our backup sets consist of a few multi-gigabyte files, so using 64K clusters will not cause much waste from slack space.

    I'd like to get your opinions on maximum disk volume sizes from a practical standpoint.

  2. #2
    Join Date
    Oct 2007
    Northampton, UK
    I'd look at running raid6 (dual parity) rather than raid5, if possible. Rebuild windows for such big volumes can be considerable. With dual parity raid you can afford to lose 2 drives.

    If you want big volumes and don't mind managing Solaris then ZFS is fantastic.

    I've setup 48TB (around 33TB usable) volumes shared over NFS with no problems.
    SysAdminMan - Asterisk PBX hosting - FreePBX, A2Billing and Elastix

Similar Threads

  1. Large Quota Master Reseller Accounts - No inode limit
    By page-zone in forum Reseller Hosting Offers
    Replies: 0
    Last Post: 06-07-2008, 04:31 AM
  2. Replies: 22
    Last Post: 01-15-2008, 11:41 AM
  3. Large File Size Limit
    By OrthATG in forum Web Hosting
    Replies: 5
    Last Post: 12-22-2005, 06:45 AM
  4. Disk Quta and bandwidth limit.
    By PrOsHoCk in forum Hosting Security and Technology
    Replies: 5
    Last Post: 11-28-2002, 03:24 PM
  5. dedicated disk partitions...
    By Steven.C in forum Dedicated Server
    Replies: 1
    Last Post: 11-16-2002, 02:26 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts