Results 1 to 25 of 29
-
10-29-2005, 10:28 PM #1Junior Guru
- Join Date
- Jun 2004
- Posts
- 237
high bandwidth advice needed. and an iowait question...
how high should iowait get before you should start to worry?
also, how fast could most servers (with just a regular hard drive, no raid or anything) serve large files? is 20mbps pushing the i/o? 50? 100?
basically things are slow right now on our current server and we're using almost 20mbps constantly. I'm wondering if upgrading the line to 50mbps will be enough to fix this or if there are more problems beyond just the bandwidth
-
10-30-2005, 05:13 AM #2Web Hosting Master
- Join Date
- Jan 2003
- Posts
- 1,715
Given a typically seek-heavy disk load, you can get around 1 Mbytes/sec from an IDE or SATA drive before it becomes a performance issue. The amount of traffic that supports depends a lot on how large your working set is.
Apache has a high per-client cost that is best left to scripts and dynamic content. I would start by using thttpd, lighttpd, boa, or squid to handle the static files. With those, 20 Mbit shouldn't be an issue for any working set. For 50 Mbit on large, varied files, you may want to add a second disk.Game Servers are the next hot market!
Slim margins, heavy support, fickle customers, and moronic suppliers!
Start your own today!
-
10-30-2005, 04:11 PM #3Junior Guru
- Join Date
- Jun 2004
- Posts
- 237
thanks. any idea on the iowait question?
-
10-31-2005, 01:15 AM #4Web Hosting Master
- Join Date
- Jan 2003
- Posts
- 1,715
For batch processing (like a stats server), it can go pretty high. For web traffic, I haven't seen it above 5-10%, and I don't think it would be pretty.
Game Servers are the next hot market!
Slim margins, heavy support, fickle customers, and moronic suppliers!
Start your own today!
-
10-31-2005, 07:30 AM #5Junior Guru Wannabe
- Join Date
- Aug 2005
- Location
- Bay Area
- Posts
- 63
1MB is VERY conservative don't you think?! Even for a system with a seek-heavy disk load...
OneZero We're a perfect 10 when it comes to your hosting and Internet needs!
To us every last customer and every server also rates a perfect ten!
http://onezerohosting.com, sales@onezerohosting.com
-
10-31-2005, 12:06 PM #6Web Hosting Guru
- Join Date
- Jun 2005
- Posts
- 300
We average about 30mbps with 0% iowait, and have maxed the 100mbps port with about 10% io wait. IDE drives
Try file hosting at Simpload.com
-
10-31-2005, 03:54 PM #7Junior Guru
- Join Date
- Jun 2004
- Posts
- 237
what's the highest iowait should be, ideally?
-
10-31-2005, 05:50 PM #8Web Hosting Guru
- Join Date
- Aug 2004
- Posts
- 346
Originally Posted by mythologen
You give to your users direct link to file or you using php for upload/download?
Thanks.
-
10-31-2005, 08:59 PM #9Web Hosting Master
- Join Date
- Jan 2003
- Posts
- 1,715
Originally Posted by alias_willsmith
gurika: You can easily get that if your traffic is mostly a few very popular files (like demo and patch release parties), because the working set fits in memory. If you have a huge, varied selection of active files, no sane amount of RAM will hold it.Game Servers are the next hot market!
Slim margins, heavy support, fickle customers, and moronic suppliers!
Start your own today!
-
10-31-2005, 09:21 PM #10Web Hosting Guru
- Join Date
- Aug 2004
- Posts
- 346
But I have SWAP 0% usage (if I good know that mean that my RAM is not 100% in use) but IOWAIT over 90% and very high load...how you put your files in RAM?
That is only in case when user downloading file with direct link to file or that can work if I use PHP to hide file location on server (PHP read and print/send file to user)...
Thanks.
-
10-31-2005, 11:56 PM #11Junior Guru
- Join Date
- Jun 2004
- Posts
- 237
I'm still wondering about how high iowait can be before it begins to impact the speed of the server
can't seem to get a straight answer for this anywhere
-
11-01-2005, 01:18 AM #12Web Hosting Master
- Join Date
- Jan 2003
- Posts
- 1,715
gurika: Sounds a bit like the RedHat kernel bug from some time ago. I haven't seen 90% iowait on any system, functional or not.
disgust: If iowait is over 10%, it will notably impact responsiveness.Game Servers are the next hot market!
Slim margins, heavy support, fickle customers, and moronic suppliers!
Start your own today!
-
11-01-2005, 08:48 PM #13Web Hosting Guru
- Join Date
- Jun 2005
- Posts
- 300
Originally Posted by disgust
Originally Posted by gurika
The hardware and software setups allow the drives to run efficiently; in other words, there is sufficient ram and processing power to aviod bottlenecks.
When we see lots of traffic it is usually generated from a small number of files being accessed a lot. If your kernel is operating well it will place these files in ram, and resolve many disk problems.Try file hosting at Simpload.com
-
11-01-2005, 09:24 PM #14Problem Solver
- Join Date
- Mar 2003
- Location
- California USA
- Posts
- 13,681
A 2.6 kernel with deadline enabled will help with iowait some.. What os are you using??
Steven Ciaburri | Industry's Best Server Management - Rack911.com
Software Auditing - 400+ Vulnerabilities Found - Quote @ https://www.RACK911Labs.com
Fully Managed Dedicated Servers (Las Vegas, New York City, & Amsterdam) (AS62710)
FreeBSD & Linux Server Management, Security Auditing, Server Optimization, PCI Compliance
-
11-01-2005, 09:52 PM #15Web Hosting Master
- Join Date
- Apr 2004
- Location
- San Jose
- Posts
- 902
IO wait is the time not allocated to system or user processing when there are uncompleted IOs.
This means you could have a very light CPU application that is doing lots of IO be very responsive with 1% user, 5% system, 80% iowait, and 14% idle. Do the same IO with more CPU, and you could see 85% user, 5% system, 0% iowait and 10%idle.
The iostat utility will give you a better idea about how your io subsystem is working for you.
Here is the current iostat result from a very PHP intensive site that is snappy:
avg-cpu: %user %nice %sys %iowait %idle
17.48 0.00 7.89 24.58 50.05
Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda 102.00 0.00 2121.04 0 10584
Notice that there is fairly high iowait, and lots of writes. That's because PHP sessions by default write files to /tmp. If this became an issue, I would create a memory filesystem and point PHP to that, or use a database backed session system.
The database is on a separate system here, so I know that's not contributing to the disk IO.
As with almost any number while doing tuning, iowait can be misleading out of context.
One thing to watch for is your idle time. If that goes to 0%, and you have significant iowait, then you are IO bound. Depending on your working set (files that get served) you should be able to improve that situation by adding RAM.
If you have sufficient idle time, then the system is able to keep ahead of the processing and IO loads.
The load numbers can be misleading. You can have lots of idle time and a load of 15 if all the jobs are very short when they get on the processor. You can have 0 idle time and a load of 1 and be dog slow if your job is very CPU intensive.Specializing in MySQL and website tuning for high traffic sites. cmwsci.com/
-
11-02-2005, 06:33 AM #16Web Hosting Guru
- Join Date
- Aug 2004
- Posts
- 346
Originally Posted by Steven
-
11-02-2005, 06:35 AM #17Web Hosting Guru
- Join Date
- Aug 2004
- Posts
- 346
Originally Posted by sailorFred
-
11-02-2005, 03:37 PM #18Web Hosting Master
- Join Date
- Apr 2004
- Location
- San Jose
- Posts
- 902
First thing to check is for disk errors in /var/log/messages, or wherever your syslog errors go.
If there's no evidence of a failing disk, then you'll want to see which processes are doing the IO.
If you do ps -el | grep ' D ' you should see which processes are in Disk Wait.
I would expect that they would be httpd instances. Then you take the PID and strace it to see what IO it's doing.
Most likely, you are using PHP sessions, which default to being stored in /tmp.
You can create a memory filesystem and ask PHP to store the session information there, if you don't have a problem with frequent reboots, or the session information is not critical.Specializing in MySQL and website tuning for high traffic sites. cmwsci.com/
-
11-03-2005, 08:43 PM #19Web Hosting Guru
- Join Date
- Aug 2004
- Posts
- 346
No erros (same problem on few servers), no PHP sessions... ps -el | grep ' D ' in 99% cases print nothing...
Maybe I have this only because I have too much downloads and HD`s can`t do all of that in same time...
Also, maybe can be problem because I store all files (thousands) in one folder??? This is stupid question but ....
-
11-03-2005, 09:23 PM #20Web Hosting Master
- Join Date
- Apr 2004
- Location
- San Jose
- Posts
- 902
In the 1% of cases there was a process in Disk Wait, what was it?
How big is your content vs. your RAM? Ideally, the most frequently hit files should fit in the disk cache in RAM.
Having lots of files in a directory can be a performance problem if the files are constantly changing. Deletions, especially, can be slow when you get into the 10,000 files/directory range.
The Apache logging will do a certain amount of disk writing.
Do iostat -k 10. Let it run for 30 seconds and paste the results. This will give us the balance between reads vs. writes and volume.Specializing in MySQL and website tuning for high traffic sites. cmwsci.com/
-
11-04-2005, 02:28 AM #21Aspiring Evangelist
- Join Date
- Nov 2003
- Posts
- 385
hiryuu is right with the 1MBytes/sec for completely *random* rw access (mysql with huge 50GB+ tables and 300+ requests/sec). Streaming data goes much faster of course, 30MBytes/sec+.
-
11-04-2005, 06:51 AM #22Newbie
- Join Date
- Oct 2005
- Location
- Sunny Spain
- Posts
- 8
Originally Posted by gurika
-
11-06-2005, 06:40 PM #23Web Hosting Guru
- Join Date
- Aug 2004
- Posts
- 346
Originally Posted by sailorFred
1 D 0 6778 1 0 75 0 - 0 io_sch ? 00:33:06 kjournald
5 D 99 28850 4561 0 75 0 - 2847 schedu ? 00:01:38 httpd
5 D 99 28851 4561 0 75 0 - 2854 schedu ? 00:01:50 httpd
5 D 99 28852 4561 0 75 0 - 2846 schedu ? 00:01:47 httpd
5 D 99 28853 4561 0 75 0 - 2851 schedu ? 00:01:53 httpd
5 D 99 28854 4561 0 75 0 - 2851 schedu ? 00:01:53 httpd
5 D 99 28856 4561 0 75 0 - 2847 schedu ? 00:01:48 httpd
5 D 99 28860 4561 0 75 0 - 2851 schedu ? 00:01:57 httpd
5 D 99 28959 4561 0 75 0 - 2851 schedu ? 00:01:50 httpd
5 D 99 28961 4561 0 75 0 - 2853 schedu ? 00:01:53 httpd
5 D 99 28967 4561 0 75 0 - 2848 io_sch ? 00:01:55 httpd
5 D 99 29038 4561 0 75 0 - 2843 schedu ? 00:01:38 httpd
5 D 99 29039 4561 0 75 0 - 2857 schedu ? 00:01:44 httpd
5 D 99 29222 4561 0 75 0 - 2858 io_sch ? 00:01:48 httpd
5 D 99 29671 4561 0 75 0 - 2848 schedu ? 00:01:48 httpd
5 D 99 29679 4561 0 75 0 - 2848 schedu ? 00:01:50 httpd
5 D 99 29901 4561 0 75 0 - 2843 schedu ? 00:02:04 httpd
5 D 99 30017 4561 0 75 0 - 2856 schedu ? 00:01:24 httpd
5 D 99 30121 4561 0 75 0 - 2844 schedu ? 00:01:23 httpd
5 D 99 30165 4561 0 75 0 - 2849 schedu ? 00:01:42 httpd
5 D 99 25273 4561 0 75 0 - 2843 schedu ? 00:01:17 httpd
5 D 99 25533 4561 0 75 0 - 2851 schedu ? 00:01:22 httpd
5 D 99 27262 4561 0 75 0 - 2853 schedu ? 00:01:09 httpd
5 D 99 27607 4561 0 75 0 - 2843 schedu ? 00:01:16 httpd
5 D 99 28015 4561 0 75 0 - 2847 schedu ? 00:01:17 httpd
5 D 99 4535 4561 0 75 0 - 2836 schedu ? 00:00:22 httpd
5 D 99 4613 4561 0 75 0 - 2833 schedu ? 00:00:26 httpd
5 D 99 4993 4561 0 75 0 - 2845 io_sch ? 00:00:17 httpd
5 D 99 5653 4561 0 75 0 - 2829 schedu ? 00:00:16 httpd
5 D 99 5672 4561 0 75 0 - 2826 schedu ? 00:00:20 httpd
5 D 99 7024 4561 0 75 0 - 2835 schedu ? 00:00:20 httpd
5 D 99 7135 4561 0 75 0 - 2833 schedu ? 00:00:14 httpd
5 D 99 7270 4561 0 75 0 - 2826 schedu ? 00:00:10 httpd
5 D 99 7438 4561 0 75 0 - 2832 io_sch ? 00:00:15 httpd
5 D 99 7439 4561 0 75 0 - 2850 schedu ? 00:00:21 httpd
5 D 99 7837 4561 0 75 0 - 2848 schedu ? 00:00:10 httpd
5 D 99 7938 4561 0 75 0 - 2835 schedu ? 00:00:17 httpd
5 D 99 7943 4561 0 75 0 - 2845 schedu ? 00:00:14 httpd
Content on server from 1KB to 60MB, RAM 2GB!
iostat -k 10 don`t work on my server: -bash: iostat: command not found
Also, you must know that I using PHP for file downoad and this is main code for that:
$fp=@fopen($file,"rb");
while (!@feof($fp) && !connection_aborted()) {
echo(@fread($fp, 8192));
}
fclose ($fp);
I don`t know is possible to store some data in RAM if I downloading files on this way, I thinking that this using very much HD and this must be main problem...
When I have very high IOWAIT my HD working very very slow:
hdparm -t /dev/sdb
/dev/sdb:
Timing buffered disk reads: 2 MB in 25.35 seconds = 80.79 kB/sec (normal over 50MB/s)
(SATA 8MB cach)
Thanks.
-
11-06-2005, 06:55 PM #24Web Hosting Guru
- Join Date
- Aug 2004
- Posts
- 346
Now I installed iostat but now this error:
iostat -k 10
iostat: invalid option -- k
and only this options:
c - print cpu usage info
d - print basic disk info
D - print disk utilization info
p - print partition info also
P - print partition info only
x - print extended disk info
h - this help
I tested this with this command:
iostat -D 10
and this is what I get:
sda sdb
r/s w/s %b r/s w/s %b
165 6 1 252 8 1
152 3 79 234 14 100
156 4 80 245 24 100
195 3 86 225 25 100
206 4 79 230 20 100
172 2 82 228 12 100
215 2 85 228 11 100
177 4 76 239 27 100
175 2 71 235 14 100
185 2 76 236 26 100
171 4 69 250 13 100
164 3 71 249 28 100
172 4 73 230 16 100
Thanks.
-
11-07-2005, 02:02 AM #25Web Hosting Master
- Join Date
- Jan 2003
- Posts
- 1,715
You really should be using readfile() or fpassthru() for sending the file.
Game Servers are the next hot market!
Slim margins, heavy support, fickle customers, and moronic suppliers!
Start your own today!