Page 2 of 3 FirstFirst 123 LastLast
Results 26 to 50 of 72
  1. #26
    Join Date
    Dec 2007
    Posts
    262
    Quote Originally Posted by funkywizard View Post
    Maybe, but there really isn't a whole lot to it:

    1) If using software raid 10, make sure the raid stripe is 2MB
    2) If using hardware raid 10, make sure the raid stripe is as large as the raid controller will allow
    3) Set the linux readahead to 1/4 of the raid stripe (if using raid 10), or to 512k (if using raid 1 or no raid at all)
    4) Disable "atimes" in /etc/fstab
    5) All else being equal, more ram, more hard drives, or faster rpm drives, are better than less ram, fewer hard drives, or slower rpm drives
    Thanks a lot for the info. Any recommendations for a software raid 0 strip size?

    thanks again.

  2. #27
    Quote Originally Posted by p2prockz View Post
    Thanks a lot for the info. Any recommendations for a software raid 0 strip size?

    thanks again.
    If you want to use raid 0 (which I don't recommend), then a 2MB stripe would also offer the best performance.
    IOFLOOD.com -- We Love Servers
    Phoenix, AZ Dedicated Servers in under an hour
    ★ Ryzen 9: 7950x3D ★ Dual E5-2680v4 Xeon ★
    Contact Us: sales@ioflood.com

  3. #28
    Join Date
    Dec 2007
    Posts
    262
    I am still figuring bits of this atop info.

    1. Under the PID for ffmpeg it says the CPU% is 427%
    Can someone explain to me what does that % means

    2. Even though ffmpeg uses 427% of CPU in this case CPL is not in critical stage. Is the CPU (E3-1230, 4 Core) powerful enough to handle the load in this instant? I thought anything above 4 should flag red?
    CPL | avg1 6.92| avg5 5.47| avg15 5.48|

    3. Also man list ('MBr/s') as the "number of MiBytes per second throughput for reads".
    In this case It is ~10 MBr/s from each 3 drives. Which means 240mb/s of data read?

    Atop: http://img155.imageshack.us/img155/9155/raid0l.png
    Last edited by p2prockz; 07-27-2012 at 08:46 PM. Reason: MBr Question

  4. #29
    Quote Originally Posted by p2prockz View Post
    I am still figuring bits of this atop info.

    1. Under the PID for ffmpeg it says the CPU% is 427%
    Can someone explain to me what does that % means

    2. Even though ffmpeg uses 427% of CPU in this case CPL is not in critical stage. Is the CPU (E3-1230, 4 Core) powerful enough to handle the load in this instant? I thought anything above 4 should flag red?
    CPL | avg1 6.92| avg5 5.47| avg15 5.48|

    3. Also man list ('MBr/s') as the "number of MiBytes per second throughput for reads".
    In this case It is ~10 MBr/s from each 3 drives. Which means 240mb/s of data read?

    Atop: http://img155.imageshack.us/img155/9155/raid0l.png
    427% refers to the percentage of one core that the program is using. In this case, it means that it is using 4 cores, plus 27% of another core. However, this doesn't tell the whole story, because modern cpus like the E3 have features such as hyperthreading, turbo boost, and speed step, which makes it very difficult to figure out the exact cpu usage.

    For example try running the following command, and paste the output here:

    cat /proc/cpuinfo | grep MHz

    If you run that command it will tell you the current clock speed of all of your cores. Even though the E3 can run at 3.2ghz, chances are that most of the cores are running much slower. The cores will automatically go faster when extra performance is needed. Turbo boost can actually make the cpu go faster than the rated speed, so it's entirely possible that even at 50% cpu usage, you're actually using as low as 20%.

    As well, you have 4 cores, but you also have hyperthreading, which shows 4 additional cores in your OS. For a floating point heavy process like ffmpeg, hyperthreading can literally double your performance. For a more integer-math intensive program, it often has little benefit. For an "average" workload, the rule of thumb is that hyperthreading gives you about a 30% performance boost, but it's impossible to tell exactly how much extra performance you will get with hyperthreading.

    Between the clock speed not staying the same at all times, and with hyperthreading giving an unpredictable performance boost, showing cpu usage of 50% could actually mean cpu usage as low as 20% or as high as 80%. The only reliable way to know for certain how much spare cpu you have, is to run your application as hard as possible, and measure how much work it is able to do. For example, if you find that you can encode 20 videos at once, at a total framerate of 1000fps, and that's the most the server can do, then if you happen to be encoding 10 videos at 500fps, then you probably have about half your cpu free, although top might not show 50% idle cpu in that case, in reality that would be how much you have left over.

    Without doing tests to see what maximum amount of work your server can handle, the cpu idle percentage doesn't tell you much. The load average value also is of extremely little use in seeing how loaded the server is. Generally my rule of thumb is, if "top" shows idle cpu of 20% or more, then you have quite a bit left. With cpu idle showing 20% or less, chances are you're very close to maxing out the performance of your server.
    IOFLOOD.com -- We Love Servers
    Phoenix, AZ Dedicated Servers in under an hour
    ★ Ryzen 9: 7950x3D ★ Dual E5-2680v4 Xeon ★
    Contact Us: sales@ioflood.com

  5. #30
    Join Date
    Dec 2007
    Posts
    262
    Quote Originally Posted by funkywizard View Post
    427% refers to the percentage of one core that the program is using. In this case, it means that it is using 4 cores, plus 27% of another core. However, this doesn't tell the whole story, because modern cpus like the E3 have features such as hyperthreading, turbo boost, and speed step, which makes it very difficult to figure out the exact cpu usage.

    For example try running the following command, and paste the output here:

    cat /proc/cpuinfo | grep MHz

    If you run that command it will tell you the current clock speed of all of your cores. Even though the E3 can run at 3.2ghz, chances are that most of the cores are running much slower. The cores will automatically go faster when extra performance is needed. Turbo boost can actually make the cpu go faster than the rated speed, so it's entirely possible that even at 50% cpu usage, you're actually using as low as 20%.

    As well, you have 4 cores, but you also have hyperthreading, which shows 4 additional cores in your OS. For a floating point heavy process like ffmpeg, hyperthreading can literally double your performance. For a more integer-math intensive program, it often has little benefit. For an "average" workload, the rule of thumb is that hyperthreading gives you about a 30% performance boost, but it's impossible to tell exactly how much extra performance you will get with hyperthreading.

    Between the clock speed not staying the same at all times, and with hyperthreading giving an unpredictable performance boost, showing cpu usage of 50% could actually mean cpu usage as low as 20% or as high as 80%. The only reliable way to know for certain how much spare cpu you have, is to run your application as hard as possible, and measure how much work it is able to do. For example, if you find that you can encode 20 videos at once, at a total framerate of 1000fps, and that's the most the server can do, then if you happen to be encoding 10 videos at 500fps, then you probably have about half your cpu free, although top might not show 50% idle cpu in that case, in reality that would be how much you have left over.

    Without doing tests to see what maximum amount of work your server can handle, the cpu idle percentage doesn't tell you much. The load average value also is of extremely little use in seeing how loaded the server is. Generally my rule of thumb is, if "top" shows idle cpu of 20% or more, then you have quite a bit left. With cpu idle showing 20% or less, chances are you're very close to maxing out the performance of your server.
    Thanks a lot for that explanation and the details would help others as well.

    Here is my output cat /proc/cpuinfo | grep MHz while ffmpeg is running (Script is configured to convert only one video at a time)

    Code:
    # cat /proc/cpuinfo | grep MHz
    cpu MHz         : 1600.000
    cpu MHz         : 1600.000
    cpu MHz         : 1600.000
    cpu MHz         : 3401.000
    cpu MHz         : 1600.000
    cpu MHz         : 1600.000
    cpu MHz         : 1600.000
    cpu MHz         : 1600.000
    So it seems CPU is not an issue (because most of them running at 1.6Ghz)?

    Here is my top output with ffmpeg running, is it showing CPU 50.45idle?

    Code:
    top - 23:36:14 up 23 days,  7:15,  1 user,  load average: 6.33, 4.91, 5.29
    Tasks: 228 total,   2 running, 226 sleeping,   0 stopped,   0 zombie
    Cpu(s): 17.8%us,  0.8%sy, 27.0%ni, 50.4%id,  3.0%wa,  0.0%hi,  1.0%si,  0.0%st
    Mem:   8152040k total,  8104484k used,    47556k free,    20748k buffers
    Swap: 10241428k total,      236k used, 10241192k free,  7315900k cached
    
      PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
    20912 root      25   0  339m 117m 3636 R 361.7  1.5  13:23.04 ffmpeg
    12427 natv    23   8  307m 228m 2692 S  2.3  2.9   3249:02 bitflu.pl
    15329 nobody    15   0 61284  13m 1636 S  2.0  0.2 462:49.92 nginx
    15328 nobody    15   0 59964  12m 1636 S  1.7  0.2 467:26.37 nginx
       11 root      RT  -5     0    0    0 S  0.3  0.0   5:27.65 migration/3
    thanks again.

  6. #31
    Looking at your output, I would say your real cpu use is around 25%, since the cores are clocked to half speed and you are using 50% of the cpu time at the reduced clock. Nornally I would say maybe 30-40% because hyperthreaded cores are not 'as good as' real cores, but for ffmpeg which is heavy on floating point, hyperthreaded cores perform nearly as well as 'real' cores. As such I would say you have plenty of cpu to spare on your current server.
    IOFLOOD.com -- We Love Servers
    Phoenix, AZ Dedicated Servers in under an hour
    ★ Ryzen 9: 7950x3D ★ Dual E5-2680v4 Xeon ★
    Contact Us: sales@ioflood.com

  7. #32
    Join Date
    Dec 2007
    Posts
    262
    Quote Originally Posted by funkywizard View Post
    Looking at your output, I would say your real cpu use is around 25%, since the cores are clocked to half speed and you are using 50% of the cpu time at the reduced clock. Nornally I would say maybe 30-40% because hyperthreaded cores are not 'as good as' real cores, but for ffmpeg which is heavy on floating point, hyperthreaded cores perform nearly as well as 'real' cores. As such I would say you have plenty of cpu to spare on your current server.
    That helps me to narrow down my search for a new server. basically i/o is the issue it seems. Will consider raid setup suggestions as well when buying the next server.

  8. #33
    Quote Originally Posted by p2prockz View Post
    That helps me to narrow down my search for a new server. basically i/o is the issue it seems. Will consider raid setup suggestions as well when buying the next server.
    Sounds good. For what it's worth, the raid stripe size is all but ignored by most hosts when installing the os. In centos, you can't set the stripe size from within the normal installer, you have to create the raid partitions from rescue mode and then have the centos installer use the partitions that were previously created. It's not surprising then that most hosts would not bother to modify this setting, which is a shame because it can boost performance four fold for use cases like yours.
    IOFLOOD.com -- We Love Servers
    Phoenix, AZ Dedicated Servers in under an hour
    ★ Ryzen 9: 7950x3D ★ Dual E5-2680v4 Xeon ★
    Contact Us: sales@ioflood.com

  9. #34
    Join Date
    Dec 2007
    Posts
    262
    I run this command on my fiel server to get current raid info.



    Code:
    root@fs4whm [~]# cat /proc/mdstat
    Personalities : [raid0]
    md127 : active raid0 sda[1] sdb[0]
          3907023872 blocks super external:/md0/0 128k chunks
    
    md0 : inactive sda[1](S) sdb[0](S)
          5032 blocks super external:imsm
    Does this means my strip size is 128kb?
    I have specifically asked them to setup "stripe size 2MB" as suggested by you guys.

  10. #35
    Yes, that output shows a 128k stripe. Unfortunately the stripe size can only be set when the raid is created :/
    IOFLOOD.com -- We Love Servers
    Phoenix, AZ Dedicated Servers in under an hour
    ★ Ryzen 9: 7950x3D ★ Dual E5-2680v4 Xeon ★
    Contact Us: sales@ioflood.com

  11. #36
    Join Date
    Dec 2007
    Posts
    262
    Thanks for that i guess they would need to reconfigure this server?

  12. #37
    Yes, they would need to reinstall the os, making sure to create the raid array before going into the os installer, because the os installer won't support modifying the stripe size. Actually, there's no excuse for a 128k stripe because the default is 256, so I don't know how they messed that one up so badly.
    IOFLOOD.com -- We Love Servers
    Phoenix, AZ Dedicated Servers in under an hour
    ★ Ryzen 9: 7950x3D ★ Dual E5-2680v4 Xeon ★
    Contact Us: sales@ioflood.com

  13. #38
    Join Date
    Dec 2007
    Posts
    262
    When i ask them about this they sent me this reply. Is he disputing this 128k strip size?

    I have submit a ticket now referring to my initial setup instructions. I now see what you meant about how lot of hosts pay no attention to RAID setup.


    Please try to install dmraid on your server.
    This utility will help you get more information about RAID arrays.
    If you need some other assistance, please ask.
    Thanks

  14. #39
    Quote Originally Posted by p2prockz View Post
    When i ask them about this they sent me this reply. Is he disputing this 128k strip size?

    I have submit a ticket now referring to my initial setup instructions. I now see what you meant about how lot of hosts pay no attention to RAID setup.
    Most technicians or customers are not aware of what stripe sizes are or why to care about them, and with how hard the os installer makes to set them, it's no surprise 99% of hosts leave the setting as default.
    IOFLOOD.com -- We Love Servers
    Phoenix, AZ Dedicated Servers in under an hour
    ★ Ryzen 9: 7950x3D ★ Dual E5-2680v4 Xeon ★
    Contact Us: sales@ioflood.com

  15. #40
    Join Date
    Dec 2007
    Posts
    262
    This is one of the popular high bandwidth providers. Now it seems he wants me to change it my self. Here is the reply

    This is a software RAID, I do not see any hardware RAID on your order. You can
    reconfigure software RAID yourself with mdadm.
    Below is some documentation that may assist you:

    http://linux.die.net/man/8/mdadm
    https://raid.wiki.kernel.org/index.php/RAID_setup

    If you have any other questions, please let us know.
    Is it possible to change this or is it their responsibility to do it since I request on setup phase?

  16. #41
    It is only possible to change the stripe size when the raid array is created, so this has to be done before the OS is installed.
    IOFLOOD.com -- We Love Servers
    Phoenix, AZ Dedicated Servers in under an hour
    ★ Ryzen 9: 7950x3D ★ Dual E5-2680v4 Xeon ★
    Contact Us: sales@ioflood.com

  17. #42
    Join Date
    Mar 2003
    Location
    Kansas City, Missouri
    Posts
    462
    Hello-

    For file hosting, why is everyone recommending RAID-10? 10 is great for database IO workloads where there is a great demand for random I/O for both reads and writes. In a file-server scenario, most of your I/O will be read related. Why not just use RAID-5 with large disks? When performing large read operations RAID-5 acts like RAID-0. Write operations will be slower but it really just depends on the I/O characteristics of the users. Will there be large data coming in or leaving the system? I imagine most of the data will be outbound (read from the array) than inbound.

    I guess it just depends on what files are hosted, how often they are read and written.

    Good luck
    =>Admo.net Managed Hosting
    => Managed Hosting • Dedicated Servers • Colocation
    => Dark Fiber Access to 1102 Grand, Multiple Public Providers
    => Over •Sixteen• Years of Service

  18. #43
    Quote Originally Posted by AdmoNet View Post
    Hello-

    For file hosting, why is everyone recommending RAID-10? 10 is great for database IO workloads where there is a great demand for random I/O for both reads and writes. In a file-server scenario, most of your I/O will be read related. Why not just use RAID-5 with large disks? When performing large read operations RAID-5 acts like RAID-0. Write operations will be slower but it really just depends on the I/O characteristics of the users. Will there be large data coming in or leaving the system? I imagine most of the data will be outbound (read from the array) than inbound.

    I guess it just depends on what files are hosted, how often they are read and written.

    Good luck
    Certainly if you're tight on disk space and you are mostly doing reads, raid 5 can work for this use case.
    IOFLOOD.com -- We Love Servers
    Phoenix, AZ Dedicated Servers in under an hour
    ★ Ryzen 9: 7950x3D ★ Dual E5-2680v4 Xeon ★
    Contact Us: sales@ioflood.com

  19. #44
    Join Date
    Sep 2005
    Location
    Albany, NY
    Posts
    3,956
    I don't really have anything else to contribute to this thread since it has been so meticulously dissected by the posters. I just wanted to praise the people, especially Gabe, who contributed to this thread. It's truly helpful, not only for the OP, but for anyone else who reads this in the future. I hope more threads like this pop up on WHT instead of the usual bashing.
    AYKsolutions.com - High Bandwidth Specialists - 10Gbps/20Gbps+ Unmetered & DDOS Protected
    Over 20+ Global Locations - Asia (Hong Kong, Singapore, Tokyo), Mexico, Brazil, India, Australia, US, CA, EU - Bare Metal and Virtual Cloud. All Managed.
    We are Professional. Painless. Polite.

  20. #45
    Join Date
    Dec 2007
    Posts
    262
    Are read and write performance for any give raid system independent of each other. For example will read performance be effected in a RAID 5 system if all of a sudden there is a spike in write (uploads)

    From my experience for a file host write is 1/5 or 1/10th of read requests at any given time (pls See attached)
    Attached Thumbnails Attached Thumbnails raid0.png  

  21. #46
    You have a certain number of i/o/s per drive to work with for both reads and writes together. With a raid 5, a read (when the stripe size is large enough and the read request small enough) will usually use one i/o operation on a single drive. A write on a raid 5 will cause both a read and a write to be performed on every drive in the array, which uses up a lot of your 'iops budget', evem with relatively few writes. A write on a raid 10 will hit two disks, two iops. On a 4 drive raid 5, a write will do two iops per drive, or 8 iops total. So raid 5 will dramatically hurt performance unless the number of writes is very near zero. Even with writes only being 10% of disk requests, they would ve taking up about 50% of all disk activity.
    IOFLOOD.com -- We Love Servers
    Phoenix, AZ Dedicated Servers in under an hour
    ★ Ryzen 9: 7950x3D ★ Dual E5-2680v4 Xeon ★
    Contact Us: sales@ioflood.com

  22. #47
    Join Date
    Dec 2007
    Posts
    262
    Thank you.

  23. #48
    Join Date
    Dec 2007
    Posts
    262
    Quote Originally Posted by funkywizard View Post
    4) Disable "atimes" in /etc/fstab
    Hi, nano /etc/fstab display follwoing information. I see noatime

    Code:
    # /etc/fstab
    # Created by anaconda on Thu Aug  2 12:12:55 2012
    #
    # Accessible filesystems, by reference, are maintained under '/dev/disk'
    # See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
    #
    UUID=f254e85b-65f3-48db-a35e-77f0b8e7caec /                       ext4    defaults        1 1
    UUID=5f566899-e8d1-4af8-b312-b12547f94b0a /boot                   ext2    defaults        1 2
    UUID=9c2cc467-f8a8-4768-8a89-dd6f6efcf405 swap                    swap    defaults        0 0
    tmpfs                   /dev/shm                tmpfs   defaults        0 0
    devpts                  /dev/pts                devpts  gid=5,mode=620  0 0
    sysfs                   /sys                    sysfs   defaults        0 0
    proc                    /proc                   proc    defaults        0 0
    /dev/md2 is the raid0
    Code:
    # blkid
    /dev/md2: UUID="f254e85b-65f3-48db-a35e-77f0b8e7caec" TYPE="ext4"
    Should I change the fstab file to something like this? Do i need all these options or just the noatime?

    Code:
    UUID=f254e85b-65f3-48db-a35e-77f0b8e7caec /               ext4    errors=remount-ro,discard,noatime,nodiratime 0

  24. #49
    Join Date
    Mar 2003
    Location
    California USA
    Posts
    13,681
    Quote Originally Posted by funkywizard View Post
    Yes, they would need to reinstall the os, making sure to create the raid array before going into the os installer, because the os installer won't support modifying the stripe size. Actually, there's no excuse for a 128k stripe because the default is 256, so I don't know how they messed that one up so badly.
    I looked at it, they used Raid sata mode and setup a intel fake raid.

  25. #50
    Quote Originally Posted by Steven View Post
    I looked at it, they used Raid sata mode and setup a intel fake raid.
    strange then that mdadm reports a raid array is present
    IOFLOOD.com -- We Love Servers
    Phoenix, AZ Dedicated Servers in under an hour
    ★ Ryzen 9: 7950x3D ★ Dual E5-2680v4 Xeon ★
    Contact Us: sales@ioflood.com

Page 2 of 3 FirstFirst 123 LastLast

Similar Threads

  1. Cant change ftp port, chkserv.d/ftpd file always back using original file
    By basketmen in forum Hosting Security and Technology
    Replies: 2
    Last Post: 02-24-2012, 12:04 AM
  2. Replies: 21
    Last Post: 08-27-2010, 07:36 PM
  3. Managed Server 10mbits-port,20mbits-port,50mbits-port,100mbits-port Unmetered
    By internetport in forum Dedicated Hosting Offers
    Replies: 0
    Last Post: 11-22-2008, 07:38 AM
  4. Dual 1GHZ P3 1GB Memory 40GB HDD CentOS/Ubuntu Server 100MBIT Port
    By seasideintl in forum Dedicated Hosting Offers
    Replies: 3
    Last Post: 04-04-2008, 05:08 PM
  5. Server suddenly Bottleneck
    By spikeyspy in forum Dedicated Server
    Replies: 2
    Last Post: 12-29-2004, 01:53 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •