Results 1 to 25 of 72
Hybrid View
-
07-18-2012, 08:36 PM #1Web Hosting Guru
- Join Date
- Dec 2007
- Posts
- 262
What is my bottleneck for file hosting using this server? HDD or Port
Hello All,
We have a read intensive video hosting service for static files. I was wondering what is going to be my bottleneck for the following configurations
Intel Xeon E3-1230 Quad Core (Should be ok for some ffmpeg tasks)
8 GB DDR3 ECC 1333 Memory
1000 GB SATA Hard Drive (Not sure what RPM or Cache, waiting reply)
1Gbps Connectivity Port
My question is without any RAID setup (I know it is risky) will i be able to utilize the full 1Gbps if need be?
Some info:
- Only 20-30Mbps will be the upload into the server (writing)
- Assume full 1Gbps is available for upload + download
-
07-18-2012, 08:41 PM #2Retired Moderator
- Join Date
- May 2004
- Location
- Toronto, Canada
- Posts
- 5,105
Disk first. A single Sata drive (or any for that matter) won't stand up to a read intensive service. You need Raid 10.
CloudNexus Technology Services
Managed Services
-
07-18-2012, 09:25 PM #3WHT Addict
- Join Date
- May 2010
- Location
- Istanbul/Turkey
- Posts
- 129
Your disk will be your bottleneck. But according to "Only 20-30Mbps will be the upload into the server (writing)" your system will work fine.
█ Hosthink
█ Dedicated Servers: 54 Locations / 6 Continents
█ Virtual Servers: Turkey & Switzerland & Finland - OpenVZ & KVM
█ Sales: sales@hosthink.net - Cryptocurrencies Accepted
-
07-18-2012, 11:10 PM #4Web Hosting Guru
- Join Date
- Dec 2007
- Posts
- 262
Assuming standard RPM and cache for the HDD can you ballpark what is the maximum upload speed I can utilize on the port?
Basically I would like to calculate the number of viewers such server will be able to handle. Thanks in advance
-
07-18-2012, 11:15 PM #5The Linux Specialist
- Join Date
- Mar 2003
- Location
- /root
- Posts
- 23,991
Hardware Raid10 is recommended when it comes to File or Video Hosting.
Specially 4 U
Reseller Hosting: Boost Your Websites | Fully Managed KVM VPS: 3.20 - 5.00 Ghz, Pure Dedicated Power
JoneSolutions.Com is on the net 24/7 providing stable and reliable web hosting solutions, server management and services since 2001
Debian|Ubuntu|cPanel|DirectAdmin|Enhance|Webuzo|Acronis|Estela|BitNinja|Nginx
-
07-18-2012, 11:18 PM #6Web Hosting Guru
- Join Date
- Dec 2007
- Posts
- 262
Not really worried about the data security since this is a free service.
my concern if I/O performance
Is a single 1TB SATA drive going to be outperformed by a 1Gbps upload port?
If so, roughly how much upload (viewers) can we do before I/O performance become and issue.
thanks
-
07-18-2012, 11:28 PM #7Attack The Day
- Join Date
- May 2010
- Location
- NYC
- Posts
- 835
NewYorkCityServers.com - Specializing In Dedicated Servers and Financial Hosting
True Enterprise Service, Tier 3 Manhattan Datacenter, 100+ Gbps Network, 100% Uptime Guarantee, 24x7 Support - Email, Tickets, Phone and Live Chat
Bandwidth Graphs, Remote Power Control, Automated OS Re-installs, Secured IPMI+KVM Included With Every Server
-
07-18-2012, 11:47 PM #8Temporarily Suspended
- Join Date
- Mar 2010
- Posts
- 1,154
To cap out a 1gbps line, assuming its unmetered you are going to need a lot more throughput. Disks can only read so fast, however the more disks you have the more throughput and reads you can have. Sounds like you are on a budget, but if you want to make full use of that I would suggest 4-8 x 500gb drives in raid-10.
-
07-18-2012, 11:54 PM #9Web Hosting Guru
- Join Date
- Dec 2007
- Posts
- 262
Thanks for the info. i have a taken atop output of my current server which has 4 HDD on a RAID0 configuration.
Whenever the servers running normal I do not see HDD on critical (red) but when I run encoding using ffmpeg all 4 drives become critical (busy ~80% to 90%)
According to atop currently server is uploading around 500Mbps and 60mbps downloading (users writing to disk)
Why is ffmpeg effecting the I/O performance? I thought it would effect the CPU performance more than I/O
atop: http://img155.imageshack.us/img155/9155/raid0l.png
If this is the case for 4 drives on RAID 0, would you say 1 drive don't stand a chance under similar load?
Thanks in advance for your help.
-
07-19-2012, 07:21 PM #10
Ironically, for serving large files with many simultaneous users, a 4 drive raid 0 is not necessarily a lot faster than a single drive alone, because the raid striping can cause a single disk read of 1MB for example, read from all 4 disks. since disks are limited in how many operations per second much moreso than they are in MB/s, having a single disk read span all 4 disks will kill your performance. If you set the stripe size large enough (I recommend a 2MB stripe and 512k linux readahead for serving large files), then a 4 drive raid 0 will perform nearly 4 times as fast as a single drive alone, but with the default stripe size of 256k and default readahead of 128k, the performance of the 4 drive raid 0 will be something crazy like 1/6th of what it could be with a 2MB stripe and 512k readahead.
IOFLOOD.com -- We Love Servers
Phoenix, AZ Dedicated Servers in under an hour
★ Ryzen 9: 7950x3D ★ Dual E5-2680v4 Xeon ★
Contact Us: sales@ioflood.com ★
-
07-21-2012, 12:12 AM #11Web Hosting Guru
- Join Date
- Dec 2007
- Posts
- 262
-
07-21-2012, 08:40 AM #12Junior Guru Wannabe
- Join Date
- Jan 2011
- Posts
- 41
to get your current readahead value:
blockdev --getra /dev/sda
to tweak readahaed value:
sudo blockdev --setra 16384 /dev/sda
change 16384 to any value (256, 512, ...,4096, 16384, 32768, 65536, 131072, 262144).
-
07-21-2012, 03:30 PM #13IOFLOOD.com -- We Love Servers
Phoenix, AZ Dedicated Servers in under an hour
★ Ryzen 9: 7950x3D ★ Dual E5-2680v4 Xeon ★
Contact Us: sales@ioflood.com ★
-
07-19-2012, 12:02 AM #14Attack The Day
- Join Date
- May 2010
- Location
- NYC
- Posts
- 835
The I/O performance of your drives are being effected when you encode a video because the video that is being encoded in stored on your hard drive. Your hard drive is being used to encode the video. Encoding a video is very resource intensive on the whole system (CPU, HDD, ect)
I would also assume that the video you are trying to encode isn't small.
A single drive would not be able to handle similar load.NewYorkCityServers.com - Specializing In Dedicated Servers and Financial Hosting
True Enterprise Service, Tier 3 Manhattan Datacenter, 100+ Gbps Network, 100% Uptime Guarantee, 24x7 Support - Email, Tickets, Phone and Live Chat
Bandwidth Graphs, Remote Power Control, Automated OS Re-installs, Secured IPMI+KVM Included With Every Server
-
07-19-2012, 12:28 AM #15Web Hosting Guru
- Join Date
- Dec 2007
- Posts
- 262
-
07-19-2012, 03:26 AM #16Web Hosting Guru
- Join Date
- Jun 2012
- Location
- Melbourne, Australia
- Posts
- 308
I would encode the videos on another drive (or box) then use Nginx/Varnish to serve the static files.
██ ServerBear - Easy 1 Command Benchmarking (IO, IOPS, UnixBench & Network)
██ Example Report - http://bit.ly/LMdmuG
██ Rankings - Current Top Performing Servers & Hosts - serverbear.com/benchmarks
██ Web Hosts - Benchmark Your Nodes & Our Performance Reports Increase Conversion Rates
-
07-19-2012, 07:16 AM #17Junior Guru Wannabe
- Join Date
- Jan 2007
- Posts
- 84
RAID 10 + ZFS = Win!
or you can try to use and SSD to cache the files...
-
07-19-2012, 08:12 AM #18Web Hosting Master
- Join Date
- Oct 2003
- Location
- The Netherlands
- Posts
- 1,270
"It depends"
As for those saying the 1 disk won't be enough to saturate the 1 Gbit line.. well that also depends on how much is cached in memory and how much will come from the disk for the current requests. (which depends on your site's usage)
If people tend to hit the same selection of files over and over and they fit in memory you'll most likely see very little IO. If the traffic is extremely random then all the requests will go to disk.
You'll also bottlneck sooner because you will be writing the converted files as well.
Raid 1/10 might be a good suggestion, or at the very least have several disks
-
07-19-2012, 08:27 AM #19Hello World
- Join Date
- Nov 2009
- Location
- /etc/my.cnf
- Posts
- 10,657
We see people attempting to use single disk setups for these type of sites time and time again I assume this is the case here again.
You need upgrade to a decent RAID-10 array sooner rather than later
Regards,UK Based Proactive Server Management.
Zabbix Enterprise 24/7 Monitoring.
-
07-27-2012, 08:42 PM #20Web Hosting Guru
- Join Date
- Dec 2007
- Posts
- 262
I am still figuring bits of this atop info.
1. Under the PID for ffmpeg it says the CPU% is 427%
Can someone explain to me what does that % means
2. Even though ffmpeg uses 427% of CPU in this case CPL is not in critical stage. Is the CPU (E3-1230, 4 Core) powerful enough to handle the load in this instant? I thought anything above 4 should flag red?
CPL | avg1 6.92| avg5 5.47| avg15 5.48|
3. Also man list ('MBr/s') as the "number of MiBytes per second throughput for reads".
In this case It is ~10 MBr/s from each 3 drives. Which means 240mb/s of data read?
Atop: http://img155.imageshack.us/img155/9155/raid0l.pngLast edited by p2prockz; 07-27-2012 at 08:46 PM. Reason: MBr Question
-
07-27-2012, 11:40 PM #21
427% refers to the percentage of one core that the program is using. In this case, it means that it is using 4 cores, plus 27% of another core. However, this doesn't tell the whole story, because modern cpus like the E3 have features such as hyperthreading, turbo boost, and speed step, which makes it very difficult to figure out the exact cpu usage.
For example try running the following command, and paste the output here:
cat /proc/cpuinfo | grep MHz
If you run that command it will tell you the current clock speed of all of your cores. Even though the E3 can run at 3.2ghz, chances are that most of the cores are running much slower. The cores will automatically go faster when extra performance is needed. Turbo boost can actually make the cpu go faster than the rated speed, so it's entirely possible that even at 50% cpu usage, you're actually using as low as 20%.
As well, you have 4 cores, but you also have hyperthreading, which shows 4 additional cores in your OS. For a floating point heavy process like ffmpeg, hyperthreading can literally double your performance. For a more integer-math intensive program, it often has little benefit. For an "average" workload, the rule of thumb is that hyperthreading gives you about a 30% performance boost, but it's impossible to tell exactly how much extra performance you will get with hyperthreading.
Between the clock speed not staying the same at all times, and with hyperthreading giving an unpredictable performance boost, showing cpu usage of 50% could actually mean cpu usage as low as 20% or as high as 80%. The only reliable way to know for certain how much spare cpu you have, is to run your application as hard as possible, and measure how much work it is able to do. For example, if you find that you can encode 20 videos at once, at a total framerate of 1000fps, and that's the most the server can do, then if you happen to be encoding 10 videos at 500fps, then you probably have about half your cpu free, although top might not show 50% idle cpu in that case, in reality that would be how much you have left over.
Without doing tests to see what maximum amount of work your server can handle, the cpu idle percentage doesn't tell you much. The load average value also is of extremely little use in seeing how loaded the server is. Generally my rule of thumb is, if "top" shows idle cpu of 20% or more, then you have quite a bit left. With cpu idle showing 20% or less, chances are you're very close to maxing out the performance of your server.IOFLOOD.com -- We Love Servers
Phoenix, AZ Dedicated Servers in under an hour
★ Ryzen 9: 7950x3D ★ Dual E5-2680v4 Xeon ★
Contact Us: sales@ioflood.com ★
-
07-28-2012, 01:38 AM #22Web Hosting Guru
- Join Date
- Dec 2007
- Posts
- 262
Thanks a lot for that explanation and the details would help others as well.
Here is my output cat /proc/cpuinfo | grep MHz while ffmpeg is running (Script is configured to convert only one video at a time)
Code:# cat /proc/cpuinfo | grep MHz cpu MHz : 1600.000 cpu MHz : 1600.000 cpu MHz : 1600.000 cpu MHz : 3401.000 cpu MHz : 1600.000 cpu MHz : 1600.000 cpu MHz : 1600.000 cpu MHz : 1600.000
Here is my top output with ffmpeg running, is it showing CPU 50.45idle?
Code:top - 23:36:14 up 23 days, 7:15, 1 user, load average: 6.33, 4.91, 5.29 Tasks: 228 total, 2 running, 226 sleeping, 0 stopped, 0 zombie Cpu(s): 17.8%us, 0.8%sy, 27.0%ni, 50.4%id, 3.0%wa, 0.0%hi, 1.0%si, 0.0%st Mem: 8152040k total, 8104484k used, 47556k free, 20748k buffers Swap: 10241428k total, 236k used, 10241192k free, 7315900k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 20912 root 25 0 339m 117m 3636 R 361.7 1.5 13:23.04 ffmpeg 12427 natv 23 8 307m 228m 2692 S 2.3 2.9 3249:02 bitflu.pl 15329 nobody 15 0 61284 13m 1636 S 2.0 0.2 462:49.92 nginx 15328 nobody 15 0 59964 12m 1636 S 1.7 0.2 467:26.37 nginx 11 root RT -5 0 0 0 S 0.3 0.0 5:27.65 migration/3
-
07-19-2012, 07:17 PM #23
Disk will be your bottleneck, although how much bandwidth you can use depends on a number of factors, most important being the size of the files you're serving. If you're serving large files (such as a video streaming site or large file host), if you bump up the linux disk readahead value to 512K (vs the default 128K), then you can get about 40MB/s from a single high quality sata drive serving files to a ton of people simultaneously, or around 300 megabit. If files are small (under 1MB for example), then the speeds will drop. The fewer total files you have, the more likely they will be in your disk cache in ram, which boosts throughput, and the more files you have, the less likely any given request will be cached in ram already, lowering throughput.
I would recommend at minimum a 2 drive software raid 1, as this will double your read speeds, and you can still benefit from increasing the linux readahead to 512k. With raid 10, sometimes the performance will be no better than raid 1, particularly for mostly-read workloads for large files, because the stripe size is normally far too small to be optimal. However, if you do set a 2MB or larger raid stripe along with a 512K readahead, then a 4 drive raid 10 should be nearly twice as fast as a 2 drive raid 1.
Regardless of the specifics, it's unlikely you'll saturate a 1gbps port using a single sata drive unless the total files you're storing are small enough that they all fit in ram. Under perfect conditions, you could get close to 1gbps with 2 drives in raid 1.IOFLOOD.com -- We Love Servers
Phoenix, AZ Dedicated Servers in under an hour
★ Ryzen 9: 7950x3D ★ Dual E5-2680v4 Xeon ★
Contact Us: sales@ioflood.com ★
-
07-24-2012, 08:27 PM #24Web Hosting Guru
- Join Date
- Dec 2007
- Posts
- 262
Hello All,
I have the option to buy a second server with the following specs
XEON E5 2603
64GB RAM
2x3TB HDD 7200
1Gbps dedicated port with 100TB transfer
They also have 2 port 3ware RAID controller. Should I go with a Hardware RAID or software for read intensive operation?
Can some one explain what is the difference between 2 and 4 port RAID controllers?
Thanks in advance
-
07-24-2012, 08:43 PM #25Web Hosting Master
- Join Date
- Oct 2002
- Location
- Vancouver, B.C.
- Posts
- 2,699
With the much larger amount of RAM, you can reduce the amount of disk I/O you'll need substantially.
1) You can create a RAM disk, and encode your video onto that first, and then just copy the completed movie to your hard drives afterwards.
2) You'll be able to cache a lot of your movies in a RAM.
Depending on the RAID card, there may be different types of ports. Either they are SATA/SAS ports, where you can only use 1 drive per port, or you can have SFF8087 ports where you can connect a 1-to-4 fan-out cable and have 4x drives per port. You'll need to clarify with the provider on which of these it is.
Assuming you're running Linux, don't bother with hardware RAID for RAID 0, 1, 10 as the RAID operations are too simple to warrant offloading to a separate card. Hardware RAID on Linux only really makes sense for RAID 5, 6, 50, 60 or other raid types where parity calculations need to be made.ASTUTE INTERNET: Advanced, customized, and scalable solutions with AS54527 Premium Performance and Canadian Optimized Network (Level3, Shaw, CogecoPeer1, GTT/Tinet),
AS63213 Cost Effective High Performance Network (Cogent, HE, GTT/Tinet)
Dedicated Hosting, Colo, Bandwidth, and Fiber out of Vancouver, Seattle, LA, Toronto, NYC, and Miami
Similar Threads
-
Cant change ftp port, chkserv.d/ftpd file always back using original file
By basketmen in forum Hosting Security and TechnologyReplies: 2Last Post: 02-24-2012, 12:04 AM -
Need a Dedicated Server: 2x Quad Core, 12-16GB RAM, 3TB+ HDD, 1Gbps Port, 20TB+ BW
By HyperVMart in forum Dedicated ServerReplies: 21Last Post: 08-27-2010, 07:36 PM -
Managed Server 10mbits-port,20mbits-port,50mbits-port,100mbits-port Unmetered
By internetport in forum Dedicated Hosting OffersReplies: 0Last Post: 11-22-2008, 07:38 AM -
Dual 1GHZ P3 1GB Memory 40GB HDD CentOS/Ubuntu Server 100MBIT Port
By seasideintl in forum Dedicated Hosting OffersReplies: 3Last Post: 04-04-2008, 05:08 PM -
Server suddenly Bottleneck
By spikeyspy in forum Dedicated ServerReplies: 2Last Post: 12-29-2004, 01:53 PM