Page 3 of 3 FirstFirst 123
Results 51 to 72 of 72
  1. #51
    Join Date
    Mar 2003
    Location
    California USA
    Posts
    13,262
    Quote Originally Posted by funkywizard View Post
    strange then that mdadm reports a raid array is present
    Yeah, I don't know. :-/

  2. #52
    Join Date
    Apr 2009
    Posts
    1,133
    Your 1tb might do around 30 megs max if It's multiple users downloadning.. If you add users uploading aswell you'll most likely end up closer to maxing a 100Mbit link.. Raid 10 for storage/serving files and a ssd disc for your encoding - imo

  3. #53
    Join Date
    Dec 2007
    Posts
    262
    Quote Originally Posted by mazedk View Post
    Your 1tb might do around 30 megs max if It's multiple users downloadning.. If you add users uploading aswell you'll most likely end up closer to maxing a 100Mbit link.. Raid 10 for storage/serving files and a ssd disc for your encoding - imo
    That was the case before funkywizard (strongly recommended if you want a host that know i/o performance) IOFLOOD.com

    and others helped me with this. If you look at my atop you can see I am doing more than 30megs.

  4. #54
    Join Date
    Apr 2009
    Posts
    1,133
    Yeah, saw that - was on my iphone in bed just browsing and a bit tired. My bad
    /maze

  5. #55
    Join Date
    Sep 2010
    Location
    Behind you...
    Posts
    355
    Interesting topic about the different raid advantages and disadvantages!
    file1.info :: 50GB secure cloudstorage with filemanager

  6. #56
    Join Date
    Dec 2007
    Posts
    262
    Hello all,

    i have gotten a new server and host agreed to install the Sodtware RAID1 (not fakeraid) with a 2MB strip size.

    Now that they have finished I get the following output

    Code:
    [root@50 ~]# cat /proc/mdstat
    Personalities : [raid1]
    md0 : active raid1 sda1[0] sdb1[1]
          511988 blocks super 1.0 [2/2] [UU]
    
    md1 : active raid1 sdb2[1] sda2[0]
          1953000316 blocks super 1.1 [2/2] [UU]
          [==>..................]  resync = 11.8% (231827264/1953000316) finish=226.8min speed=126443K/sec
          bitmap: 14/15 pages [56KB], 65536KB chunk
    
    unused devices: <none>
    What have they done now!
    1. Isn't this chunk size way to bigger than the requested 2MB?
    2. also what is the resync and bitmap info say about this setup?


    Thanks in advance

  7. #57
    Quote Originally Posted by p2prockz View Post
    Hello all,

    i have gotten a new server and host agreed to install the Sodtware RAID1 (not fakeraid) with a 2MB strip size.

    Now that they have finished I get the following output

    Code:
    [root@50 ~]# cat /proc/mdstat
    Personalities : [raid1]
    md0 : active raid1 sda1[0] sdb1[1]
          511988 blocks super 1.0 [2/2] [UU]
    
    md1 : active raid1 sdb2[1] sda2[0]
          1953000316 blocks super 1.1 [2/2] [UU]
          [==>..................]  resync = 11.8% (231827264/1953000316) finish=226.8min speed=126443K/sec
          bitmap: 14/15 pages [56KB], 65536KB chunk
    
    unused devices: <none>
    What have they done now!
    1. Isn't this chunk size way to bigger than the requested 2MB?
    2. also what is the resync and bitmap info say about this setup?


    Thanks in advance
    Raid 1 doesn't have a stripe size, because there is no striping, only mirroring. I'm not sure why it says 65MB chunk, that's kind of a misleading piece of information there.
    Phoenix Dedicated Servers -- IOFLOOD.com
    Email: sales [at] ioflood.com
    Skype: iofloodsales
    Backup Storage VPS -- 1TBVPS.com

  8. #58
    Join Date
    Dec 2007
    Posts
    262
    Quote Originally Posted by funkywizard View Post
    Raid 1 doesn't have a stripe size, because there is no striping, only mirroring. I'm not sure why it says 65MB chunk, that's kind of a misleading piece of information there.

    Is there any specific raid 1 setup that would help in our case (video hosting for large 100mb and up files)?

    Thanks again.

  9. #59
    Quote Originally Posted by p2prockz View Post
    Is there any specific raid 1 setup that would help in our case (video hosting for large 100mb and up files)?

    Thanks again.
    The only configuration for raid 1 that would be helpful is to increase the linux readahead value to 512k, which you can do with the following command:

    blockdev --setra 1024 /dev/sda
    blockdev --setra 1024 /dev/sdb
    blockdev --setra 1024 /dev/sda1
    blockdev --setra 1024 /dev/sda1
    blockdev --setra 1024 /dev/sda2
    blockdev --setra 1024 /dev/sda2
    blockdev --setra 1024 /dev/md0
    blockdev --setra 1024 /dev/md1

    etc (do this essentially for each partition and each raid volume)

    The above won't "stick" between reboots, so you can make this change permanent by editing /etc/rc.d/rc.local and adding those commands above to it.
    Phoenix Dedicated Servers -- IOFLOOD.com
    Email: sales [at] ioflood.com
    Skype: iofloodsales
    Backup Storage VPS -- 1TBVPS.com

  10. #60
    Join Date
    Dec 2007
    Posts
    262
    Quote Originally Posted by funkywizard View Post
    The only configuration for raid 1 that would be helpful is to increase the linux readahead value to 512k, which you can do with the following command:

    blockdev --setra 1024 /dev/sda
    blockdev --setra 1024 /dev/sdb
    blockdev --setra 1024 /dev/sda1
    blockdev --setra 1024 /dev/sda1
    blockdev --setra 1024 /dev/sda2
    blockdev --setra 1024 /dev/sda2
    blockdev --setra 1024 /dev/md0
    blockdev --setra 1024 /dev/md1

    etc (do this essentially for each partition and each raid volume)

    The above won't "stick" between reboots, so you can make this change permanent by editing /etc/rc.d/rc.local and adding those commands above to it.
    Thanks for the help!

  11. #61
    I accidnetally duplicated sda1 and sda2 twice. One should be sda1 / sda2 and another should be sdb1 / sdb2
    Phoenix Dedicated Servers -- IOFLOOD.com
    Email: sales [at] ioflood.com
    Skype: iofloodsales
    Backup Storage VPS -- 1TBVPS.com

  12. #62
    Join Date
    Mar 2003
    Posts
    352
    Sorry for jumping in, assuming i am only serving a few numbers of files, and the total size of all files (100MB) fits well within the system Memory capacity (512MB).

    Then could i assume that all file will be served from cache / Ram?
    And therefore speed of Disk does not come into the equation?

    Would CPU speed ever become a bottleneck? Let say i am only running on a Single Core Atom?

  13. #63
    Quote Originally Posted by ayksolutions View Post
    I don't really have anything else to contribute to this thread since it has been so meticulously dissected by the posters. I just wanted to praise the people, especially Gabe, who contributed to this thread. It's truly helpful, not only for the OP, but for anyone else who reads this in the future. I hope more threads like this pop up on WHT instead of the usual bashing.
    I concur; reading this thread was very informative. Thanks everyone for your contribution here.

  14. #64
    Join Date
    Dec 2007
    Posts
    262
    Hello, I am back here to get some of your expertise regarding a RAID 10 setup.

    It was pointed out that stripe size of 2MB is recommended for a software RAID 10 and for Hardware RAID10 we should go with the maximum stripe size allowed by the controller.

    I have an issue where I paid $$ for a HW raid controller which only support a max stripe size of 256K

    What should I do in this case to get the best performance?
    Should I cancel the Hardware RAID controller and go with software raid?

    Thanks in advance.

  15. #65
    Quote Originally Posted by p2prockz View Post
    Hello, I am back here to get some of your expertise regarding a RAID 10 setup.

    It was pointed out that stripe size of 2MB is recommended for a software RAID 10 and for Hardware RAID10 we should go with the maximum stripe size allowed by the controller.

    I have an issue where I paid $$ for a HW raid controller which only support a max stripe size of 256K

    What should I do in this case to get the best performance?
    Should I cancel the Hardware RAID controller and go with software raid?

    Thanks in advance.
    If you're primarily working with large files with sequential access, then definitely yes. For large numbers of simultaneous reads of large files, a 2MB stripe and a 512k readahead will provide at least 4x the performance of a 256k stripe.
    Phoenix Dedicated Servers -- IOFLOOD.com
    Email: sales [at] ioflood.com
    Skype: iofloodsales
    Backup Storage VPS -- 1TBVPS.com

  16. #66
    Join Date
    Dec 2007
    Posts
    262
    Thanks a lot for the reply. Wish they had a thanks button for profiles.

  17. #67
    Glad to help : )
    Phoenix Dedicated Servers -- IOFLOOD.com
    Email: sales [at] ioflood.com
    Skype: iofloodsales
    Backup Storage VPS -- 1TBVPS.com

  18. #68
    Join Date
    Jul 2012
    Location
    Europe
    Posts
    21
    Wow, It depends on many things.. Its very hard for me to answer right away, I might get back to this one..

  19. #69
    Join Date
    Dec 2007
    Posts
    262

    RAID 10 resync

    Hi All, Had setup software raid10 and I see the partition is re-syncing. Is resyncing done in the case of repairing the partition? or is it normal for it to run after the raid10 installation for the first time?



    Code:
    [root@server ~]# cat /proc/mdstat
    Personalities : [raid10] [raid1]
    md0 : active raid1 sda1[0] sdc1[2] sdd1[3] sdb1[1]
          487360 blocks [4/4] [UUUU]
    
    md2 : active raid10 sdc3[2] sda3[0] sdb3[1] sdd3[3]
          5843927040 blocks super 1.2 2048K chunks 2 near-copies [4/4] [UUUU]
          [===>.................]  resync = 15.3% (896459072/5843927040) finish=74870.8min speed=1100K/sec
    
    md1 : active raid10 sdc2[2] sda2[0] sdb2[1] sdd2[3]
          15621120 blocks super 1.1 512K chunks 2 near-copies [4/4] [UUUU]
    
    unused devices: <none>
    What kind of commands can I use to get the status related to RAID10? Can I get information directly from the HW controller? I am very new to RAID so any reading material would be helpful.

    Thanks.

  20. #70
    Join Date
    Dec 2007
    Posts
    262
    Quote Originally Posted by funkywizard View Post
    Maybe, but there really isn't a whole lot to it:

    1) If using software raid 10, make sure the raid stripe is 2MB
    2) If using hardware raid 10, make sure the raid stripe is as large as the raid controller will allow
    3) Set the linux readahead to 1/4 of the raid stripe (if using raid 10), or to 512k (if using raid 1 or no raid at all)
    4) Disable "atimes" in /etc/fstab
    5) All else being equal, more ram, more hard drives, or faster rpm drives, are better than less ram, fewer hard drives, or slower rpm drives

    Hello again,

    Just wondering would it be beneficial to set a stripe size even larger than 2MB? For example, since we have files larger than 100Mb can we set the strip size to let's say 4MB or 8MB?

    Thanks.

  21. #71
    Quote Originally Posted by p2prockz View Post
    Hello again,

    Just wondering would it be beneficial to set a stripe size even larger than 2MB? For example, since we have files larger than 100Mb can we set the strip size to let's say 4MB or 8MB?

    Thanks.
    There is no additional performance benefit when you use a readahead larger than 512KB, and in fact you can lose performance in some cases because you'll have to read the same data more than once if your readaheads are taking up too much ram. With a 2MB stripe and a 512k readahead, approximately 25% of your read requests will cross a stripe boundary, so 75% of the time, one read request causes one disk i/o on one drive, and 25% of the time you'll get one disk i/o on each of two drives. A 4mb stripe should drop this to 12.5%. So for 10 reads, instead of 12.5 i/o requests, you'd have 11.25 i/o requests. So potentially a 4MB stripe might be a little faster in this use pattern than a 2MB stripe, but it's a pretty small difference and there might be negative performance from having the stripe be too large, so I'd say 2MB is a safe bet. You could use 4MB if you want, I would expect that to be fine too, but I haven't tested that. Anything above 4MB when you have a 512k readahead won't do much for you. A readahead above 512k won't do much for you either.
    Phoenix Dedicated Servers -- IOFLOOD.com
    Email: sales [at] ioflood.com
    Skype: iofloodsales
    Backup Storage VPS -- 1TBVPS.com

  22. #72
    Join Date
    Dec 2007
    Posts
    262
    Thank you again!

  23. Newsletters

    Subscribe Now & Get The WHT Quick Start Guide!

Page 3 of 3 FirstFirst 123

Similar Threads

  1. Cant change ftp port, chkserv.d/ftpd file always back using original file
    By basketmen in forum Hosting Security and Technology
    Replies: 2
    Last Post: 02-24-2012, 12:04 AM
  2. Replies: 21
    Last Post: 08-27-2010, 07:36 PM
  3. Replies: 0
    Last Post: 11-22-2008, 07:38 AM
  4. Dual 1GHZ P3 1GB Memory 40GB HDD CentOS/Ubuntu Server 100MBIT Port
    By seasideintl in forum Dedicated Hosting Offers
    Replies: 3
    Last Post: 04-04-2008, 05:08 PM
  5. Server suddenly Bottleneck
    By spikeyspy in forum Dedicated Server
    Replies: 2
    Last Post: 12-29-2004, 01:53 PM

Related Posts from theWHIR.com

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •