Page 1 of 2 12 LastLast
Results 1 to 25 of 37
  1. #1
    Join Date
    Mar 2011
    Location
    QC, Canada
    Posts
    49

    Help determining optimal stripe size for RAID-10


    What is the recommended stripe size for Windows Server 2008 R2 x64 on a RAID-10 (4x WD RE4 500GB).

    My RAID card: Adaptec 6405



    Array #1 (80GB should be enough ?): for Windows Server 2008 R2 x64 + SQL Server 2008 R2 x64 ?
    Recommended stripe size: ?


    Array #2 (850GB): db + small backup + the majority of the stored files will be less than 2MB
    Recommended stripe size: ?


    Thank you

  2. #2
    I typically recommend a stripe size large enough that most single disk accesses will only "hit" one of your drives. Since these are windows, I'm not totally sure how windows handles readaheads on requests, or what the request size would be, but generally, the larger the stripe size the better for any random i/o pattern you would typically see in a server environment.

    Personally, I do at least a 1MB stripe, preferably 2MB if it's available, whenever possible. There's little or no downside to having the stripe be that big in most or all access patterns, and in many typical access patterns, a stripe that big will give you a huge performance boost compared to, say, a 64-256k stripe size.
    IOFLOOD.com -- We Love Servers
    Phoenix, AZ Dedicated Servers in under an hour
    ★ Ryzen 9: 7950x3D ★ Dual E5-2680v4 Xeon ★
    Contact Us: sales@ioflood.com

  3. #3
    Join Date
    Mar 2011
    Location
    QC, Canada
    Posts
    49
    Thanks funkywizard for you advise. I did post in the wrong section.

    Moderator, can you delete this entire topic please ?

  4. #4
    Join Date
    Sep 2008
    Location
    Dallas, TX
    Posts
    4,568
    Quote Originally Posted by funkywizard View Post
    I typically recommend a stripe size large enough that most single disk accesses will only "hit" one of your drives. Since these are windows, I'm not totally sure how windows handles readaheads on requests, or what the request size would be, but generally, the larger the stripe size the better for any random i/o pattern you would typically see in a server environment.

    Personally, I do at least a 1MB stripe, preferably 2MB if it's available, whenever possible. There's little or no downside to having the stripe be that big in most or all access patterns, and in many typical access patterns, a stripe that big will give you a huge performance boost compared to, say, a 64-256k stripe size.
    Now this is where I fall short in my knowledge, why do a lot of places do stripes in the size of 64k-256k and not just use 2MB? Is there a reason behind it or is it something from years ago when that was "normal" and "good" that has just since stuck around?
    Jacob Wall

  5. #5
    Join Date
    Mar 2011
    Location
    QC, Canada
    Posts
    49
    Quote Originally Posted by funkywizard View Post
    1MB stripe, preferably 2MB if it's available
    Yes, both are available but you are sure that is not too much, i won't host big file on my server !

  6. #6
    Quote Originally Posted by Jacob Wall View Post
    Now this is where I fall short in my knowledge, why do a lot of places do stripes in the size of 64k-256k and not just use 2MB? Is there a reason behind it or is it something from years ago when that was "normal" and "good" that has just since stuck around?
    Yes, it's a throwback for the most part. In a raid 5, there are slight benefits for doing a smaller stripe because it makes parity calculations and writes a little quicker, but for a raid 10, there's not much point.

    Quote Originally Posted by Cyberr View Post
    Yes, both are available but you are sure that is not too much, i won't host big file on my server !
    There's a much bigger problem from making the stripe too small than too big. If you can read an entire file from one drive, then you get twice the i/o/s than you would get if the read request has to hit more than one drive to complete. Therefore, you want the stripe size to be 2-4 times the size of a typical read request. In linux, this would be 2-4 times the readahead value. In windows, I'm not sure how they handle readaheads, but I can't see any situation where there would be significant performance degradation on a raid 10 with a stripe that's "too big".
    IOFLOOD.com -- We Love Servers
    Phoenix, AZ Dedicated Servers in under an hour
    ★ Ryzen 9: 7950x3D ★ Dual E5-2680v4 Xeon ★
    Contact Us: sales@ioflood.com

  7. #7
    Join Date
    Mar 2011
    Location
    QC, Canada
    Posts
    49
    @funkywizard:

    Per your recommendation, I will do 1MB stripe (2MB not available).

    Any recommendation if i should do one big Array of 930GB or two arrays mentionned below ?


    RAID-10 Array #1 (80GB): for Windows Server 2008 R2 x64 + SQL Server 2008 R2 x64
    stripe size: 1MB


    RAID-10 Array #2 (850GB): db + small backup + the majority of the stored files will be less than 2MB
    stripe size: 1MB




    Thanks for your help.
    Last edited by CD Burnt; 05-06-2011 at 12:29 PM.

  8. #8
    Whether to use one array or two is really a matter of personal preference. I feel using one big array gives me the most flexibility in how I use my disk space, but there's not a huge reason in my mind to recommend one vs the other.
    IOFLOOD.com -- We Love Servers
    Phoenix, AZ Dedicated Servers in under an hour
    ★ Ryzen 9: 7950x3D ★ Dual E5-2680v4 Xeon ★
    Contact Us: sales@ioflood.com

  9. #9
    Join Date
    Mar 2011
    Location
    QC, Canada
    Posts
    49
    @funkywizard:

    OK and i have also to choose a cluster size ? Which size ?

  10. #10
    Quote Originally Posted by Cyberr View Post
    @funkywizard:

    OK and i have also to choose a cluster size ? Which size ?
    The cluster size is probably not that important, from what I remember, but I'll leave that question open to other people who may know better than I do.
    IOFLOOD.com -- We Love Servers
    Phoenix, AZ Dedicated Servers in under an hour
    ★ Ryzen 9: 7950x3D ★ Dual E5-2680v4 Xeon ★
    Contact Us: sales@ioflood.com

  11. #11
    Join Date
    Mar 2011
    Location
    QC, Canada
    Posts
    49

    RAID-10 with 1MB stripe size

    @funkywizard :

    What do you think of my results (RAID-10 1MB stripe). Should i expect to more or i'm ok ?



    -----------------------------------------------------------------------
    CrystalDiskMark 3.0.1 x64 (C) 2007-2010 hiyohiyo
    -----------------------------------------------------------------------
    * MB/s = 1,000,000 byte/s [SATA/300 = 300,000,000 byte/s]

    Sequential Read : 429.463 MB/s
    Sequential Write : 279.062 MB/s
    Random Read 512KB : 214.244 MB/s
    Random Write 512KB : 205.231 MB/s
    Random Read 4KB (QD=1) : 2.378 MB/s [ 580.6 IOPS]
    Random Write 4KB (QD=1) : 3.002 MB/s [ 733.0 IOPS]
    Random Read 4KB (QD=32) : 12.283 MB/s [ 2998.8 IOPS]
    Random Write 4KB (QD=32) : 3.235 MB/s [ 789.8 IOPS]

    Test : 500 MB [C: 18.7% (18.7/100.0 GB)] (x2)
    Date : 2011/05/18 22:52:53
    OS : Windows Server 2008 R2 Server Standard Edition (full installation) SP1 [6.1 Build 7601] (x64)



    -----------------------------------------------------------------------
    CrystalDiskMark 3.0.1 x64 (C) 2007-2010 hiyohiyo
    -----------------------------------------------------------------------
    * MB/s = 1,000,000 byte/s [SATA/300 = 300,000,000 byte/s]

    Sequential Read : 502.272 MB/s
    Sequential Write : 287.794 MB/s
    Random Read 512KB : 61.802 MB/s
    Random Write 512KB : 170.346 MB/s
    Random Read 4KB (QD=1) : 1.018 MB/s [ 248.4 IOPS]
    Random Write 4KB (QD=1) : 3.167 MB/s [ 773.2 IOPS]
    Random Read 4KB (QD=32) : 5.982 MB/s [ 1460.5 IOPS]
    Random Write 4KB (QD=32) : 3.119 MB/s [ 761.4 IOPS]

    Test : 1000 MB [C: 18.7% (18.7/100.0 GB)] (x2)
    Date : 2011/05/18 22:36:38
    OS : Windows Server 2008 R2 Server Standard Edition (full installation) SP1 [6.1 Build 7601] (x64)



    Attached Thumbnails Attached Thumbnails 500MB.jpg   1000MB.jpg  

  12. #12
    looks like a good result to me for that number of those kind of drives.
    IOFLOOD.com -- We Love Servers
    Phoenix, AZ Dedicated Servers in under an hour
    ★ Ryzen 9: 7950x3D ★ Dual E5-2680v4 Xeon ★
    Contact Us: sales@ioflood.com

  13. #13
    Join Date
    Oct 2004
    Posts
    632
    Quote Originally Posted by Jacob Wall View Post
    Now this is where I fall short in my knowledge, why do a lot of places do stripes in the size of 64k-256k and not just use 2MB? Is there a reason behind it or is it something from years ago when that was "normal" and "good" that has just since stuck around?
    I have some question too, why people nowadays are still using 125 or 256K?

  14. #14
    Quote Originally Posted by PeterPP View Post
    I have some question too, why people nowadays are still using 125 or 256K?
    The pros / cons of various stripe sizes isn't exactly common knowledge, is the only reason I can come up with.

    The only other thing, is that on a desktop computer (rather than a server), you might see a performance benefit from a smaller stripe size, as it would speed up the reading of small to medium sized files, assuming there were only ever one file being read from or written to at any given time. This is because a file which is larger than the stripe size will be read from multiple drives, where a smaller file is not. When you are only reading one file at a time, you would prefer to be reading from all of the drives simultaneously. However, even with a 1M stripe, large files will still be read from all drives. Only files under 1M would see any benefit from the smaller stripe size, and even then, only if you assume there is no queue of files to be read, that they all are read one after another in their entirety. This obviously doesn't come close to matching a typical server environment.
    IOFLOOD.com -- We Love Servers
    Phoenix, AZ Dedicated Servers in under an hour
    ★ Ryzen 9: 7950x3D ★ Dual E5-2680v4 Xeon ★
    Contact Us: sales@ioflood.com

  15. #15
    Join Date
    Oct 2004
    Posts
    632
    I know your theory, but why still majority of "server" hosting companies still use 128/256K strip size? is it due to less trouble or being more stable or just don;t know benefits of bigger strip?

  16. #16
    Join Date
    Dec 2007
    Location
    Indiana, USA
    Posts
    19,196
    This is something we debated for a while and we ended up going with the default setting for the card which I believe was 64kb. Most files on the server are small so it's not caused major issues but we will be testing various stripe sizes in detail on our newer hardware.

    I could see how larger would have less potentially drawbacks than smaller, so we may just opt for 1 MB+ but we will need to do some testing.
    Michael Denney - MDDHosting.com - Proudly hosting more than 37,700 websites since 2007.
    Ultra-Fast Cloud Shared and Pay-By-Use Reseller Hosting Powered by LiteSpeed!
    cPanel • Free SSL • 100% Uptime SLA • 24/7 Support
    Class-leading support that responds in minutes, not days.

  17. #17
    For software raid, during the centos installer, there is no way that I could find to set the raid stripe to anything other than the default 256k. So that certainly makes doing a different stripe size a challenge in that usage case, where you're setting up the raid on a boot drive. For hardware raid, a lot of cards don't support a larger stripe. And then of course, there's general ignorance. It's not exactly common knowledge, stripe sizes and readaheads and whatnot. And finally, if customers aren't asking for it, why give it?

    Ironically, the same kinds of places that will spend big bucks on SAS drives with twice the speed and 1/10th the space, won't know enough to change stripe sizes or readahead. I had one person I know running a file hosting site where they literally got a 10 fold increase in disk performance by changing those two parameters. Previous to this change, they had been primarily using SSD, but also had some SAS servers (which were overloaded). After making the change, it would have been unnecessary to have the SSD servers at all, and the SAS would have been perfectly fine, or even SATA probably (although I don't know if they did get rid of the SSDs or not).
    IOFLOOD.com -- We Love Servers
    Phoenix, AZ Dedicated Servers in under an hour
    ★ Ryzen 9: 7950x3D ★ Dual E5-2680v4 Xeon ★
    Contact Us: sales@ioflood.com

  18. #18
    Quote Originally Posted by MikeDVB View Post
    This is something we debated for a while and we ended up going with the default setting for the card which I believe was 64kb. Most files on the server are small so it's not caused major issues but we will be testing various stripe sizes in detail on our newer hardware.

    I could see how larger would have less potentially drawbacks than smaller, so we may just opt for 1 MB+ but we will need to do some testing.
    For raid 5, there can also be a drawback for writes, because it has to recalculate the parity based on the data in an entire stripe. So a write that's not a full stripe in size will entail a read of the entire stripe on all drives in order to calculate the parity, and then a write for both the data you're writing and the parity. So for raid 5, small writes will not perform very well with a larger than necessary stripe size. Even so, I would still recommend a stripe in the 256-512k range even for a raid 5.

    I'm fairly certain a raid 10 doesn't have these issues, as all data is simply mirrored to another drive, so writes to partial stripes shouldn't cause an unnecessary read request, nor will it have to read or write a larger amount of data than the data you're actually requesting be read or written, regardless of the stripe size. So for raid 10, the only case where performance would be worse is for sequentially accessing a large number of small-medium files, with no random i/o and no simultaneous i/o. I can't really think of any use cases where that would be the case aside from maybe booting an operating system, so for a server environment, it's really a no brainer. Worst case, it's not going to help your performance, but it certainly won't hurt it either.
    IOFLOOD.com -- We Love Servers
    Phoenix, AZ Dedicated Servers in under an hour
    ★ Ryzen 9: 7950x3D ★ Dual E5-2680v4 Xeon ★
    Contact Us: sales@ioflood.com

  19. #19
    Join Date
    Dec 2007
    Location
    Indiana, USA
    Posts
    19,196
    Quote Originally Posted by funkywizard View Post
    For raid 5, there can also be a drawback for writes, because it has to recalculate the parity based on the data in an entire stripe. So a write that's not a full stripe in size will entail a read of the entire stripe on all drives in order to calculate the parity, and then a write for both the data you're writing and the parity. So for raid 5, small writes will not perform very well with a larger than necessary stripe size. Even so, I would still recommend a stripe in the 256-512k range even for a raid 5.
    Indeed, we run RAID6 on our backup server which deals with extremely large files only so we went ahead with a larger stripe size there. We're talking files in the 200 GB+ range with VERY little under 1MB beyond some operating system files, etc.

    Quote Originally Posted by funkywizard View Post
    I'm fairly certain a raid 10 doesn't have these issues, as all data is simply mirrored to another drive, so writes to partial stripes shouldn't cause an unnecessary read request, nor will it have to read or write a larger amount of data than the data you're actually requesting be read or written, regardless of the stripe size. So for raid 10, the only case where performance would be worse is for sequentially accessing a large number of small-medium files, with no random i/o and no simultaneous i/o. I can't really think of any use cases where that would be the case aside from maybe booting an operating system, so for a server environment, it's really a no brainer. Worst case, it's not going to help your performance, but it certainly won't hurt it either.
    Indeed, when I did look into it I was really unable to find any real solid information on it and most of what I did find was very out-dated. By the time I thought to test the stripe size as apposed to simply accepting the default we already have several busy servers running with a smaller stripe size.

    They all perform well and without issues, I/O isn't a major issue - so it's not a major concern at this point however if we were experiencing I/O issues then it would be a major concern.

    This isn't to say we won't be provisioning future servers with larger stripe sizes. Your previous message stated that you wanted the stripe size to be larger than the read-ahead by 2~4, or did I misunderstand?

    You just seem to have a lot more experience with this than myself and I haven't had the opportunity to really tinker/test it so don't mind me if I pick your brain a bit
    Michael Denney - MDDHosting.com - Proudly hosting more than 37,700 websites since 2007.
    Ultra-Fast Cloud Shared and Pay-By-Use Reseller Hosting Powered by LiteSpeed!
    cPanel • Free SSL • 100% Uptime SLA • 24/7 Support
    Class-leading support that responds in minutes, not days.

  20. #20
    Join Date
    Mar 2011
    Location
    QC, Canada
    Posts
    49
    Quote Originally Posted by funkywizard View Post
    looks like a good result to me for that number of those kind of drives.
    Thanks for the useful help you gave me.

    I hope this thread will help other people.

  21. #21
    Quote Originally Posted by MikeDVB View Post
    This isn't to say we won't be provisioning future servers with larger stripe sizes. Your previous message stated that you wanted the stripe size to be larger than the read-ahead by 2~4, or did I misunderstand?
    Yes, having the stripe be 4x the readahead value should certainly be sufficient. In Linux, the default readahead is 128k, so a stripe of 512k would be sufficient if you do not increase the default readahead.

    Now, what also comes into play, is if you have a kind of server that reads many large files simultaneously (like a tube site or file host), then the default 128k readahead is also insufficient. I've found that a 512k readahead in linux will give you 4 times the disk i/o when you have a large number of simultaneous users downloading 4MB files at 500kilobit/s each (simulating a tube site), giving you 40MB/s performance vs the 10MB/s performance you get with a 128K readahead, on a single 1tb western digital black drive without raid. Anything above 512k readahead provided no additional benefit.

    Now, where this comes in, is that a 512K readahead will read from multiple disks simultaneously, for 100% of read requests on a raid, unless your raid stripe is 2-4 times the readahead value. Since the optimal readahead value is 512K, this means the optimal stripe size is 2MB, which would mean that you would only have a single read request hit two disks 25% of the time. A 1M stripe would be marginally acceptable if that's the maximum value your raid controller can accept, but 2MB would be preferable. And of course, if you can't use at least a 1MB stripe, then you should set your linux readahead lower, so that your read requests don't hit multiple disks for no good reason, as the performance benefit you get from the larger readahead would be offset by the performance impact of reading from multiple disks for every request, making the whole thing a wash.
    IOFLOOD.com -- We Love Servers
    Phoenix, AZ Dedicated Servers in under an hour
    ★ Ryzen 9: 7950x3D ★ Dual E5-2680v4 Xeon ★
    Contact Us: sales@ioflood.com

  22. #22
    Quote Originally Posted by Cyberr View Post
    Thanks for the useful help you gave me.

    I hope this thread will help other people.
    Glad to be able to help.
    IOFLOOD.com -- We Love Servers
    Phoenix, AZ Dedicated Servers in under an hour
    ★ Ryzen 9: 7950x3D ★ Dual E5-2680v4 Xeon ★
    Contact Us: sales@ioflood.com

  23. #23
    Join Date
    Mar 2009
    Posts
    3,700
    Hi,a shared hosting server with php/mysql,how much the stripe size is better for raid 10 ? thanx

  24. #24
    Join Date
    May 2004
    Location
    Toronto, Canada
    Posts
    5,105
    After a lot of research the best balance we came up with was actually 64k stripe. I will try to find the whitepapers that recommended it.

  25. #25
    Join Date
    Jan 2010
    Location
    so cal
    Posts
    234
    Well on our RAID-10 setup we use 256, but I guess it all depends what you're running on them. Web hosting in our case.

Page 1 of 2 12 LastLast

Similar Threads

  1. Replies: 12
    Last Post: 05-18-2011, 03:58 PM
  2. What stripe size do you use?
    By Joel Theodore in forum Dedicated Server
    Replies: 2
    Last Post: 10-04-2009, 07:47 AM
  3. Determining the font size of a tag based on popularity?
    By anlene in forum Programming Discussion
    Replies: 1
    Last Post: 04-29-2006, 04:27 PM
  4. optimal RAID stripe size?
    By john2k in forum Hosting Security and Technology
    Replies: 0
    Last Post: 11-07-2005, 06:27 PM
  5. FTP client: Determining the actual directory size
    By Ganah ALLAN in forum Hosting Security and Technology
    Replies: 5
    Last Post: 03-31-2003, 09:06 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •