Page 1 of 3 123 LastLast
Results 1 to 25 of 72
  1. #1
    Join Date
    Dec 2007
    Posts
    262

    What is my bottleneck for file hosting using this server? HDD or Port

    Hello All,
    We have a read intensive video hosting service for static files. I was wondering what is going to be my bottleneck for the following configurations

    Intel Xeon E3-1230 Quad Core (Should be ok for some ffmpeg tasks)
    8 GB DDR3 ECC 1333 Memory
    1000 GB SATA Hard Drive (Not sure what RPM or Cache, waiting reply)
    1Gbps Connectivity Port

    My question is without any RAID setup (I know it is risky) will i be able to utilize the full 1Gbps if need be?

    Some info:
    - Only 20-30Mbps will be the upload into the server (writing)
    - Assume full 1Gbps is available for upload + download

  2. #2
    Join Date
    May 2004
    Location
    Toronto, Canada
    Posts
    5,105
    Disk first. A single Sata drive (or any for that matter) won't stand up to a read intensive service. You need Raid 10.
    CloudNexus Technology Services
    Managed Services

  3. #3
    Join Date
    May 2010
    Location
    Istanbul/Turkey
    Posts
    129
    Your disk will be your bottleneck. But according to "Only 20-30Mbps will be the upload into the server (writing)" your system will work fine.
    Hosthink
    Dedicated Servers: 54 Locations / 6 Continents
    Virtual Servers: Turkey & Switzerland & Finland - OpenVZ & KVM
    Sales: sales@hosthink.net - Cryptocurrencies Accepted

  4. #4
    Join Date
    Dec 2007
    Posts
    262
    Assuming standard RPM and cache for the HDD can you ballpark what is the maximum upload speed I can utilize on the port?

    Basically I would like to calculate the number of viewers such server will be able to handle. Thanks in advance

  5. #5
    Join Date
    Mar 2003
    Location
    /root
    Posts
    23,990
    Hardware Raid10 is recommended when it comes to File or Video Hosting.

    Specially 4 U
    Reseller Hosting: Boost Your Websites | Fully Managed KVM VPS: 3.20 - 5.00 Ghz, Pure Dedicated Power
    JoneSolutions.Com is on the net 24/7 providing stable and reliable web hosting solutions, server management and services since 2001
    Debian|Ubuntu|cPanel|DirectAdmin|Enhance|Webuzo|Acronis|Estela|BitNinja|Nginx

  6. #6
    Join Date
    Dec 2007
    Posts
    262
    Not really worried about the data security since this is a free service.
    my concern if I/O performance

    Is a single 1TB SATA drive going to be outperformed by a 1Gbps upload port?
    If so, roughly how much upload (viewers) can we do before I/O performance become and issue.

    thanks

  7. #7
    Quote Originally Posted by p2prockz View Post
    Not really worried about the data security since this is a free service.
    my concern if I/O performance

    Is a single 1TB SATA drive going to be outperformed by a 1Gbps upload port?
    If so, roughly how much upload (viewers) can we do before I/O performance become and issue.

    thanks
    You will not be able to use the 1Gbps port fully if you just use a single 1TB drive.
    There is not a specific number of users before I/O will become a problem. There are a few factors to take into account. One of which is how resource intensive your website script is.
    NewYorkCityServers.com - Specializing In Dedicated Servers and Financial Hosting
    True Enterprise Service, Tier 3 Manhattan Datacenter, 100+ Gbps Network, 100% Uptime Guarantee, 24x7 Support - Email, Tickets, Phone and Live Chat
    Bandwidth Graphs, Remote Power Control, Automated OS Re-installs, Secured IPMI+KVM Included With Every Server

  8. #8
    Join Date
    Mar 2010
    Posts
    1,154
    To cap out a 1gbps line, assuming its unmetered you are going to need a lot more throughput. Disks can only read so fast, however the more disks you have the more throughput and reads you can have. Sounds like you are on a budget, but if you want to make full use of that I would suggest 4-8 x 500gb drives in raid-10.

  9. #9
    Join Date
    Dec 2007
    Posts
    262
    Thanks for the info. i have a taken atop output of my current server which has 4 HDD on a RAID0 configuration.

    Whenever the servers running normal I do not see HDD on critical (red) but when I run encoding using ffmpeg all 4 drives become critical (busy ~80% to 90%)

    According to atop currently server is uploading around 500Mbps and 60mbps downloading (users writing to disk)

    Why is ffmpeg effecting the I/O performance? I thought it would effect the CPU performance more than I/O

    atop: http://img155.imageshack.us/img155/9155/raid0l.png

    If this is the case for 4 drives on RAID 0, would you say 1 drive don't stand a chance under similar load?

    Thanks in advance for your help.
    Attached Thumbnails Attached Thumbnails raid0.png  

  10. #10
    The I/O performance of your drives are being effected when you encode a video because the video that is being encoded in stored on your hard drive. Your hard drive is being used to encode the video. Encoding a video is very resource intensive on the whole system (CPU, HDD, ect)

    I would also assume that the video you are trying to encode isn't small.

    A single drive would not be able to handle similar load.
    NewYorkCityServers.com - Specializing In Dedicated Servers and Financial Hosting
    True Enterprise Service, Tier 3 Manhattan Datacenter, 100+ Gbps Network, 100% Uptime Guarantee, 24x7 Support - Email, Tickets, Phone and Live Chat
    Bandwidth Graphs, Remote Power Control, Automated OS Re-installs, Secured IPMI+KVM Included With Every Server

  11. #11
    Join Date
    Dec 2007
    Posts
    262
    Quote Originally Posted by NYCServers-Nick View Post
    The I/O performance of your drives are being effected when you encode a video because the video that is being encoded in stored on your hard drive. Your hard drive is being used to encode the video. Encoding a video is very resource intensive on the whole system (CPU, HDD, ect)

    I would also assume that the video you are trying to encode isn't small.

    A single drive would not be able to handle similar load.
    Yes the video is about 200-300Mb.
    Is it possible to use a different drive on the same system just for encoding? i.e. specify ffmpeg to use a SSD drive attached to the system just for this purpose?

    Any thoughts about this?

  12. #12
    Join Date
    Jun 2012
    Location
    Melbourne, Australia
    Posts
    308
    I would encode the videos on another drive (or box) then use Nginx/Varnish to serve the static files.
    ██ ServerBear - Easy 1 Command Benchmarking (IO, IOPS, UnixBench & Network)
    ██ Example Report - http://bit.ly/LMdmuG
    ██ Rankings - Current Top Performing Servers & Hosts - serverbear.com/benchmarks
    ██ Web Hosts - Benchmark Your Nodes & Our Performance Reports Increase Conversion Rates

  13. #13
    Join Date
    Jan 2007
    Posts
    84
    RAID 10 + ZFS = Win!

    or you can try to use and SSD to cache the files...

  14. #14
    Join Date
    Oct 2003
    Location
    The Netherlands
    Posts
    1,270
    "It depends"
    As for those saying the 1 disk won't be enough to saturate the 1 Gbit line.. well that also depends on how much is cached in memory and how much will come from the disk for the current requests. (which depends on your site's usage)

    If people tend to hit the same selection of files over and over and they fit in memory you'll most likely see very little IO. If the traffic is extremely random then all the requests will go to disk.
    You'll also bottlneck sooner because you will be writing the converted files as well.

    Raid 1/10 might be a good suggestion, or at the very least have several disks

  15. #15
    Join Date
    Nov 2009
    Location
    /etc/my.cnf
    Posts
    10,657
    We see people attempting to use single disk setups for these type of sites time and time again I assume this is the case here again.

    You need upgrade to a decent RAID-10 array sooner rather than later

    Regards,
    UK Based Proactive Server Management.
    Zabbix Enterprise 24/7 Monitoring.

  16. #16
    Quote Originally Posted by p2prockz View Post
    Hello All,
    We have a read intensive video hosting service for static files. I was wondering what is going to be my bottleneck for the following configurations

    Intel Xeon E3-1230 Quad Core (Should be ok for some ffmpeg tasks)
    8 GB DDR3 ECC 1333 Memory
    1000 GB SATA Hard Drive (Not sure what RPM or Cache, waiting reply)
    1Gbps Connectivity Port

    My question is without any RAID setup (I know it is risky) will i be able to utilize the full 1Gbps if need be?

    Some info:
    - Only 20-30Mbps will be the upload into the server (writing)
    - Assume full 1Gbps is available for upload + download
    Disk will be your bottleneck, although how much bandwidth you can use depends on a number of factors, most important being the size of the files you're serving. If you're serving large files (such as a video streaming site or large file host), if you bump up the linux disk readahead value to 512K (vs the default 128K), then you can get about 40MB/s from a single high quality sata drive serving files to a ton of people simultaneously, or around 300 megabit. If files are small (under 1MB for example), then the speeds will drop. The fewer total files you have, the more likely they will be in your disk cache in ram, which boosts throughput, and the more files you have, the less likely any given request will be cached in ram already, lowering throughput.

    I would recommend at minimum a 2 drive software raid 1, as this will double your read speeds, and you can still benefit from increasing the linux readahead to 512k. With raid 10, sometimes the performance will be no better than raid 1, particularly for mostly-read workloads for large files, because the stripe size is normally far too small to be optimal. However, if you do set a 2MB or larger raid stripe along with a 512K readahead, then a 4 drive raid 10 should be nearly twice as fast as a 2 drive raid 1.

    Regardless of the specifics, it's unlikely you'll saturate a 1gbps port using a single sata drive unless the total files you're storing are small enough that they all fit in ram. Under perfect conditions, you could get close to 1gbps with 2 drives in raid 1.
    IOFLOOD.com -- We Love Servers
    Phoenix, AZ Dedicated Servers in under an hour
    ★ Ryzen 9: 7950x3D ★ Dual E5-2680v4 Xeon ★
    Contact Us: sales@ioflood.com

  17. #17
    Quote Originally Posted by p2prockz View Post
    Thanks for the info. i have a taken atop output of my current server which has 4 HDD on a RAID0 configuration.

    Whenever the servers running normal I do not see HDD on critical (red) but when I run encoding using ffmpeg all 4 drives become critical (busy ~80% to 90%)

    According to atop currently server is uploading around 500Mbps and 60mbps downloading (users writing to disk)

    Why is ffmpeg effecting the I/O performance? I thought it would effect the CPU performance more than I/O

    atop: http://img155.imageshack.us/img155/9155/raid0l.png

    If this is the case for 4 drives on RAID 0, would you say 1 drive don't stand a chance under similar load?

    Thanks in advance for your help.
    Ironically, for serving large files with many simultaneous users, a 4 drive raid 0 is not necessarily a lot faster than a single drive alone, because the raid striping can cause a single disk read of 1MB for example, read from all 4 disks. since disks are limited in how many operations per second much moreso than they are in MB/s, having a single disk read span all 4 disks will kill your performance. If you set the stripe size large enough (I recommend a 2MB stripe and 512k linux readahead for serving large files), then a 4 drive raid 0 will perform nearly 4 times as fast as a single drive alone, but with the default stripe size of 256k and default readahead of 128k, the performance of the 4 drive raid 0 will be something crazy like 1/6th of what it could be with a 2MB stripe and 512k readahead.
    IOFLOOD.com -- We Love Servers
    Phoenix, AZ Dedicated Servers in under an hour
    ★ Ryzen 9: 7950x3D ★ Dual E5-2680v4 Xeon ★
    Contact Us: sales@ioflood.com

  18. #18
    Join Date
    Dec 2007
    Posts
    262
    Quote Originally Posted by funkywizard View Post
    Ironically, for serving large files with many simultaneous users, a 4 drive raid 0 is not necessarily a lot faster than a single drive alone, because the raid striping can cause a single disk read of 1MB for example, read from all 4 disks. since disks are limited in how many operations per second much moreso than they are in MB/s, having a single disk read span all 4 disks will kill your performance. If you set the stripe size large enough (I recommend a 2MB stripe and 512k linux readahead for serving large files), then a 4 drive raid 0 will perform nearly 4 times as fast as a single drive alone, but with the default stripe size of 256k and default readahead of 128k, the performance of the 4 drive raid 0 will be something crazy like 1/6th of what it could be with a 2MB stripe and 512k readahead.
    Thanks for the information/suggestions by everybody here. It really helps to have your expertise so I know what questions to ask a DC and what to buy next. Thanks again

    Do you know how I can retrieve this information from SSH
    stripe size/ readahead ?

  19. #19
    Join Date
    Jan 2011
    Posts
    41
    Quote Originally Posted by p2prockz View Post
    ..Do you know how I can retrieve this information from SSH
    stripe size/ readahead ?
    to get your current readahead value:
    blockdev --getra /dev/sda
    Shows you (in 512-byte blocks) the system readahead.

    to tweak readahaed value:
    sudo blockdev --setra 16384 /dev/sda
    notice the "--setra" switch.
    change 16384 to any value (256, 512, ...,4096, 16384, 32768, 65536, 131072, 262144).

  20. #20
    Quote Originally Posted by giridara View Post
    to get your current readahead value:

    Shows you (in 512-byte blocks) the system readahead.

    to tweak readahaed value:


    notice the "--setra" switch.
    change 16384 to any value (256, 512, ...,4096, 16384, 32768, 65536, 131072, 262144).
    Beat me to it!

    A 512k readahead is optimal for large file serving, IF your stripe size is at least 2-4x as big (1M-2M). A 512k readahead can be gotten with a 1024 blocks readahead (each block being 512 bytes).
    IOFLOOD.com -- We Love Servers
    Phoenix, AZ Dedicated Servers in under an hour
    ★ Ryzen 9: 7950x3D ★ Dual E5-2680v4 Xeon ★
    Contact Us: sales@ioflood.com

  21. #21
    Join Date
    Dec 2007
    Posts
    262
    Hello All,

    I have the option to buy a second server with the following specs

    XEON E5 2603
    64GB RAM
    2x3TB HDD 7200
    1Gbps dedicated port with 100TB transfer

    They also have 2 port 3ware RAID controller. Should I go with a Hardware RAID or software for read intensive operation?

    Can some one explain what is the difference between 2 and 4 port RAID controllers?

    Thanks in advance

  22. #22
    Join Date
    Oct 2002
    Location
    Vancouver, B.C.
    Posts
    2,699
    Quote Originally Posted by p2prockz View Post
    Hello All,

    I have the option to buy a second server with the following specs

    XEON E5 2603
    64GB RAM
    2x3TB HDD 7200
    1Gbps dedicated port with 100TB transfer

    They also have 2 port 3ware RAID controller. Should I go with a Hardware RAID or software for read intensive operation?

    Can some one explain what is the difference between 2 and 4 port RAID controllers?

    Thanks in advance
    With the much larger amount of RAM, you can reduce the amount of disk I/O you'll need substantially.
    1) You can create a RAM disk, and encode your video onto that first, and then just copy the completed movie to your hard drives afterwards.
    2) You'll be able to cache a lot of your movies in a RAM.

    Depending on the RAID card, there may be different types of ports. Either they are SATA/SAS ports, where you can only use 1 drive per port, or you can have SFF8087 ports where you can connect a 1-to-4 fan-out cable and have 4x drives per port. You'll need to clarify with the provider on which of these it is.

    Assuming you're running Linux, don't bother with hardware RAID for RAID 0, 1, 10 as the RAID operations are too simple to warrant offloading to a separate card. Hardware RAID on Linux only really makes sense for RAID 5, 6, 50, 60 or other raid types where parity calculations need to be made.
    ASTUTE INTERNET: Advanced, customized, and scalable solutions with AS54527 Premium Performance and Canadian Optimized Network (Level3, Shaw, CogecoPeer1, GTT/Tinet),
    AS63213 Cost Effective High Performance Network (Cogent, HE, GTT/Tinet)
    Dedicated Hosting, Colo, Bandwidth, and Fiber out of Vancouver, Seattle, LA, Toronto, NYC, and Miami

  23. #23
    Join Date
    Dec 2007
    Posts
    262
    Quote Originally Posted by hhw View Post
    With the much larger amount of RAM, you can reduce the amount of disk I/O you'll need substantially.
    1) You can create a RAM disk, and encode your video onto that first, and then just copy the completed movie to your hard drives afterwards.
    2) You'll be able to cache a lot of your movies in a RAM.

    Depending on the RAID card, there may be different types of ports. Either they are SATA/SAS ports, where you can only use 1 drive per port, or you can have SFF8087 ports where you can connect a 1-to-4 fan-out cable and have 4x drives per port. You'll need to clarify with the provider on which of these it is.

    Assuming you're running Linux, don't bother with hardware RAID for RAID 0, 1, 10 as the RAID operations are too simple to warrant offloading to a separate card. Hardware RAID on Linux only really makes sense for RAID 5, 6, 50, 60 or other raid types where parity calculations need to be made.

    thanks a lot for that information.
    Do you know if I need to setup ffmpeg differently in order to take advantage of this RAM?

    Basically from what I hear on this thread I have to make sure ffmpeg and nginx(which we use to deliver video) take advantage of this RAM. Does it make sense to hire someone that knows I/O to optimize this server?

  24. #24
    Join Date
    Oct 2002
    Location
    Vancouver, B.C.
    Posts
    2,699
    Quote Originally Posted by p2prockz View Post
    thanks a lot for that information.
    Do you know if I need to setup ffmpeg differently in order to take advantage of this RAM?

    Basically from what I hear on this thread I have to make sure ffmpeg and nginx(which we use to deliver video) take advantage of this RAM. Does it make sense to hire someone that knows I/O to optimize this server?
    You just need to create a RAM disk, which will just reserve the specified amount of RAM and it will logically behave just like a hard drive. You then just have to choose it as the disk location to encode the movie to.

    The Linux kernel will automatically cache popular files from disk into memory, so you don't have to do anything with Nginx. You will have more control on how things get cached if you setup Nginx to do caching though, which might help.
    ASTUTE INTERNET: Advanced, customized, and scalable solutions with AS54527 Premium Performance and Canadian Optimized Network (Level3, Shaw, CogecoPeer1, GTT/Tinet),
    AS63213 Cost Effective High Performance Network (Cogent, HE, GTT/Tinet)
    Dedicated Hosting, Colo, Bandwidth, and Fiber out of Vancouver, Seattle, LA, Toronto, NYC, and Miami

  25. #25
    Quote Originally Posted by p2prockz View Post
    Does it make sense to hire someone that knows I/O to optimize this server?
    Maybe, but there really isn't a whole lot to it:

    1) If using software raid 10, make sure the raid stripe is 2MB
    2) If using hardware raid 10, make sure the raid stripe is as large as the raid controller will allow
    3) Set the linux readahead to 1/4 of the raid stripe (if using raid 10), or to 512k (if using raid 1 or no raid at all)
    4) Disable "atimes" in /etc/fstab
    5) All else being equal, more ram, more hard drives, or faster rpm drives, are better than less ram, fewer hard drives, or slower rpm drives
    IOFLOOD.com -- We Love Servers
    Phoenix, AZ Dedicated Servers in under an hour
    ★ Ryzen 9: 7950x3D ★ Dual E5-2680v4 Xeon ★
    Contact Us: sales@ioflood.com

Page 1 of 3 123 LastLast

Similar Threads

  1. Cant change ftp port, chkserv.d/ftpd file always back using original file
    By basketmen in forum Hosting Security and Technology
    Replies: 2
    Last Post: 02-24-2012, 12:04 AM
  2. Replies: 21
    Last Post: 08-27-2010, 07:36 PM
  3. Managed Server 10mbits-port,20mbits-port,50mbits-port,100mbits-port Unmetered
    By internetport in forum Dedicated Hosting Offers
    Replies: 0
    Last Post: 11-22-2008, 07:38 AM
  4. Dual 1GHZ P3 1GB Memory 40GB HDD CentOS/Ubuntu Server 100MBIT Port
    By seasideintl in forum Dedicated Hosting Offers
    Replies: 3
    Last Post: 04-04-2008, 05:08 PM
  5. Server suddenly Bottleneck
    By spikeyspy in forum Dedicated Server
    Replies: 2
    Last Post: 12-29-2004, 01:53 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •