FastCGI problems (apache 2.2, php 5.1, suexec)

I have just switched from mod_php to fastcgi + suexec, but now trying to use the ab command times out :
Quote:


ab -n 10000 -c 100 http://www.example.com/
This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright 2006 The Apache Software Foundation, http://www.apache.org/
Benchmarking www.example.com (be patient)
Completed 1000 requests
Completed 2000 requests
Completed 3000 requests
Completed 4000 requests
Completed 5000 requests
Completed 6000 requests
Completed 7000 requests
apr_poll: The timeout specified has expired (70007)
Total of 7422 requests completed


Then I get a bunch of emails from LFD
Quote:


Onderwerp: lfd on server.example.com: Excessive processes running under user example
Time: Wed Dec 17 05:08:48 2008 -0600
Account: example
Process Count: 61 (Not killed)
Process Information:
User:example PID:4515 Run Time:51(secs) Memory:181096(kb) exe:/usr/bin/php-cgi cmd:/usr/bin/php-cgi
User:example PID:4516 Run Time:51(secs) Memory:181096(kb) exe:/usr/bin/php-cgi cmd:/usr/bin/php-cgi
User:example PID:4517 Run Time:51(secs) Memory:181084(kb) exe:/usr/bin/php-cgi cmd:/usr/bin/php-cgi
User:example PID:4518 Run Time:51(secs) Memory:181092(kb) exe:/usr/bin/php-cgi cmd:/usr/bin/php-cgi
User:example PID:4523 Run Time:50(secs) Memory:181084(kb) exe:/usr/bin/php-cgi cmd:/usr/bin/php-cgi
User:example PID:4524 Run Time:50(secs) Memory:181096(kb) exe:/usr/bin/php-cgi cmd:/usr/bin/php-cgi
User:example PID:4525 Run Time:50(secs) Memory:181092(kb) exe:/usr/bin/php-cgi cmd:/usr/bin/php-cgi
User:example PID:4526 Run Time:50(secs) Memory:181084(kb) exe:/usr/bin/php-cgi cmd:/usr/bin/php-cgi
User:example PID:4527 Run Time:50(secs) Memory:181084(kb) exe:/usr/bin/php-cgi cmd:/usr/bin/php-cgi
User:example PID:4528 Run Time:50(secs) Memory:181088(kb) exe:/usr/bin/php-cgi cmd:/usr/bin/php-cgi
User:example PID:4540 Run Time:50(secs) Memory:181092(kb) exe:/usr/bin/php-cgi cmd:/usr/bin/php-cgi


(and about 50 more such lines)
Also a ton of these emais:
Quote:


Onderwerp: lfd on server.example.com: Excessive resource usage: example (8269)
Time: Wed Dec 17 05:09:58 2008 -0600
Account: example
Resource: Virtual Memory Size
Exceeded: 176 > 100 (MB)
Executable: /usr/bin/php-cgi
Command Line: /usr/bin/php-cgi
PID: 8269
Killed: No


So what's going on here? Any ideas? I knew fastcgi would use more memory.. but this?
I then made some changes to the wrapper, and removed PHP_FCGI_CHILDREN as some people said that it is bad to use it?
Well, I do get less timeouts now, they happen every few times I try the ab command though. Here is an ab that succeeded:
Quote:


[root@server ~]# ab -n 10000 -c 100 http://www.example.com/
This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright 2006 The Apache Software Foundation, http://www.apache.org/
Benchmarking www.example.com (be patient)
Completed 1000 requests
Completed 2000 requests
Completed 3000 requests
Completed 4000 requests
Completed 5000 requests
Completed 6000 requests
Completed 7000 requests
Completed 8000 requests
Completed 9000 requests
Finished 10000 requests
Server Software: Apache
Server Hostname: www.example.com
Server Port: 80
Document Path: /
Document Length: 6724 bytes
Concurrency Level: 100
Time taken for tests: 51.277226 seconds
Complete requests: 10000
Failed requests: 20
(Connect: 0, Length: 20, Exceptions: 0)
Write errors: 0
Non-2xx responses: 20
Total transferred: 69465480 bytes
HTML transferred: 67116200 bytes
Requests per second: 195.02 [#/sec] (mean)
Time per request: 512.772 [ms] (mean)
Time per request: 5.128 [ms] (mean, across all concurrent requests)
Transfer rate: 1322.95 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.4 0 7
Processing: 0 489 4464.7 0 51032
Waiting: 0 489 4464.7 0 51032
Total: 0 489 4464.7 0 51032
Percentage of the requests served within a certain time (ms)
50% 0
66% 0
75% 17
80% 18
90% 32
95% 37
98% 67
99% 13000
100% 51032 (longest request)


See 99 and 100%, why are those taking so long in comparison to the rest?
Is it normal that this stuff is so high and so much during an ab test:
Quote:


Real memory: 3.86 GB total / 1.70 GB free Swap space: 8 GB total / 7.98 GB free
18024 apache 866880 kB /usr/sbin/httpd.worker <--- 896MB ?...
17872 apache 801488 kB /usr/sbin/httpd.worker
17838 apache 80**** kB /usr/sbin/httpd.worker
2669 mysql 419436 kB /usr/libexec/mysqld --basedir=/usr --datadir=/var/lib/mysql --user=mysql --pid-f ...
18166 apache 409768 kB /usr/sbin/httpd.worker
18164 apache 409756 kB /usr/sbin/httpd.worker
3141 root 240748 kB /usr/bin/python -tt /usr/sbin/yum-updatesd
7528 example 223900 kB /usr/bin/php-cgi <--- 223MB per php-cgi process? (well is this some sort of virtual memory since obviously it can't be really all that big)
7520 example 223068 kB /usr/bin/php-cgi
8203 example 215388 kB /usr/bin/php-cgi
8653 example 215388 kB /usr/bin/php-cgi
9550 example 215388 kB /usr/bin/php-cgi
10357 example 215388 kB /usr/bin/php-cgi
10446 example 215388 kB /usr/bin/php-cgi
10895 example 215388 kB /usr/bin/php-cgi
11078 example 215388 kB /usr/bin/php-cgi
11440 example 215388 kB /usr/bin/php-cgi
11702 example 215388 kB /usr/bin/php-cgi
11956 example 215388 kB /usr/bin/php-cgi
12743 example 215388 kB /usr/bin/php-cgi
13009 example 215388 kB /usr/bin/php-cgi
13532 example 215388 kB /usr/bin/php-cgi
13928 example 215388 kB /usr/bin/php-cgi
14436 example 215388 kB /usr/bin/php-cgi
8296 example 215384 kB /usr/bin/php-cgi
8566 example 215384 kB /usr/bin/php-cgi
8739 example 215384 kB /usr/bin/php-cgi
9730 example 215384 kB /usr/bin/php-cgi
10990 example 215384 kB /usr/bin/php-cgi
11746 example 215384 kB /usr/bin/php-cgi
11831 example 215384 kB /usr/bin/php-cgi
12219 example 215384 kB /usr/bin/php-cgi
13100 example 215384 kB /usr/bin/php-cgi
13187 example 215384 kB /usr/bin/php-cgi
13449 example 215384 kB /usr/bin/php-cgi
14095 example 215384 kB /usr/bin/php-cgi
15038 example 215384 kB /usr/bin/php-cgi
15380 example 215384 kB /usr/bin/php-cgi
7684 example 215380 kB /usr/bin/php-cgi
7686 example 215380 kB /usr/bin/php-cgi
7795 example 215380 kB /usr/bin/php-cgi
9188 example 215380 kB /usr/bin/php-cgi
9818 example 215380 kB /usr/bin/php-cgi
9993 example 215380 kB /usr/bin/php-cgi
10626 example 215380 kB /usr/bin/php-cgi
10709 example 215380 kB /usr/bin/php-cgi
11350 example 215380 kB /usr/bin/php-cgi
12306 example 215380 kB /usr/bin/php-cgi
12656 example 215380 kB /usr/bin/php-cgi
13204 example 215380 kB /usr/bin/php-cgi
13838 example 215380 kB /usr/bin/php-cgi
14011 example 215380 kB /usr/bin/php-cgi
14525 example 215380 kB /usr/bin/php-cgi
14859 example 215380 kB /usr/bin/php-cgi
15036 example 215380 kB /usr/bin/php-cgi
15296 example 215380 kB /usr/bin/php-cgi
12044 example 215376 kB /usr/bin/php-cgi
13360 example 215376 kB /usr/bin/php-cgi
7687 example 215264 kB /usr/bin/php-cgi
7690 example 215264 kB /usr/bin/php-cgi
8162 example 215264 kB /usr/bin/php-cgi
9006 example 215264 kB /usr/bin/php-cgi
9371 example 215264 kB /usr/bin/php-cgi
9462 example 215264 kB /usr/bin/php-cgi
10084 example 215264 kB /usr/bin/php-cgi
11612 example 215264 kB /usr/bin/php-cgi
12389 example 215264 kB /usr/bin/php-cgi
12831 example 215264 kB /usr/bin/php-cgi
12920 example 215264 kB /usr/bin/php-cgi
13667 example 215264 kB /usr/bin/php-cgi
14246 example 215264 kB /usr/bin/php-cgi
7691 example 215260 kB /usr/bin/php-cgi
7692 example 215260 kB /usr/bin/php-cgi
7694 example 215260 kB /usr/bin/php-cgi
8833 example 215260 kB /usr/bin/php-cgi
10267 example 215260 kB /usr/bin/php-cgi
11265 example 215260 kB /usr/bin/php-cgi
14260 example 215260 kB /usr/bin/php-cgi
14703 example 215260 kB /usr/bin/php-cgi
15129 example 215260 kB /usr/bin/php-cgi
7685 example 215256 kB /usr/bin/php-cgi
7693 example 215256 kB /usr/bin/php-cgi
7981 example 215256 kB /usr/bin/php-cgi
8070 example 215256 kB /usr/bin/php-cgi
8474 example 215256 kB /usr/bin/php-cgi
8920 example 215256 kB /usr/bin/php-cgi
9098 example 215256 kB /usr/bin/php-cgi
9277 example 215256 kB /usr/bin/php-cgi
10171 example 215256 kB /usr/bin/php-cgi
12484 example 215256 kB /usr/bin/php-cgi
12514 example 215256 kB /usr/bin/php-cgi
13755 example 215256 kB /usr/bin/php-cgi
14349 example 215256 kB /usr/bin/php-cgi
14611 example 215256 kB /usr/bin/php-cgi
14946 example 215256 kB /usr/bin/php-cgi
7688 example 215252 kB /usr/bin/php-cgi
7888 example 215252 kB /usr/bin/php-cgi
8386 example 215252 kB /usr/bin/php-cgi
9640 example 215252 kB /usr/bin/php-cgi
9907 example 215252 kB /usr/bin/php-cgi
10532 example 215252 kB /usr/bin/php-cgi
10806 example 215252 kB /usr/bin/php-cgi
11173 example 215252 kB /usr/bin/php-cgi
11526 example 215252 kB /usr/bin/php-cgi
12134 example 215252 kB /usr/bin/php-cgi
13547 example 215252 kB /usr/bin/php-cgi
15467 example 215252 kB /usr/bin/php-cgi
6842 apache 128268 kB /usr/sbin/httpd.worker
5778 root 128232 kB /usr/sbin/httpd.worker <-- is this normal (root) ?


These stay loaded even after the ab has finished running. (hours later, still there) This was a server that had 300MB ram in use when using mod_php..
After checking the error log, these are the errors i found:
in /home/example/logs/error_log
Quote:


[Wed Dec 17 10:29:57 2008] [warn] (104)Connection reset by peer: mod_fcgid: read data from fastcgi server error.
[Wed Dec 17 10:29:57 2008] [error] [client 69.65.x.x] Premature end of script headers: index.php


(multiple times)
in /var/log/httpd/error_log
Quote:


[Wed Dec 17 10:30:01 2008] [notice] mod_fcgid: call /home/example/public_html/index.php with wrapper /home/example/fcgi-bin/php5.fcgi
[Wed Dec 17 10:30:10 2008] [notice] mod_fcgid: process /home/example/public_html/index.php(7693) exit(normal exit), terminated by calling exit(), return code: 0


(multiple times)
Quote:


[Wed Dec 17 10:30:10 2008] [warn] mod_fcgid: cleanup zombie process 7693


Here are my config files:
My fcgi wrapper:
Code:
#!/bin/sh
PHPRC=$PWD/../etc/php5
export PHPRC
umask 022
SCRIPT_FILENAME=$PATH_TRANSLATED
export SCRIPT_FILENAME
exec /usr/bin/php-cgi
fcgid.conf (apache)
Code:
AddHandler fcgid-script .php

# Sane place to put sockets and shared memory file
SocketPath run/mod_fcgid
SharememPath run/fcgid_shm

IdleTimeout 3600
ProcessLifeTime 7200
IPCConnectTimeout 30
IPCCommTimeout 45
MaxRequestsPerProcess 500


Virtualhost:
Code:
SuexecUserGroup example example

...

AddHandler fcgid-script .php
FCGIWrapper /home/example/fcgi-bin/php5.fcgi .php

...
Any help is appreciated, thanks.