
|
View Full Version : Huge httpd processes and very high loads
I just got a new dedicated server (2 days ago). I have nothing on it. Just Cpanel/WHM.
The server is a RedHat P4 2.4Gig with 1 gig ram and 2 80gig HD with software RAID1
Im getting enourmous loads on the server and found that the httpd processes are huge and getting bigger with time:
1:27pm up 2 days, 22:06, 1 user, load average: 9.39, 5.21, 3.60
80 processes: 74 sleeping, 6 running, 0 zombie, 0 stopped
CPU states: 1.5% user, 0.3% system, 0.0% nice, 98.0% idle
Mem: 1032132K av, 852444K used, 179688K free, 0K shrd, 19060K buff
Swap: 2096376K av, 0K used, 2096376K free 619844K cached
PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND
767 nobody 15 0 1204 1204 948 S 0.0 0.1 0:00 proftpd
903 nobody 17 0 2012 2012 1344 S 0.0 0.1 0:00 entropychat
908 nobody 15 0 872 872 592 S 0.0 0.0 0:00 melange
20722 nobody 15 0 435M 435M 435M S 0.0 43.1 0:00 httpd
20723 nobody 15 0 435M 435M 435M S 0.0 43.1 0:00 httpd
20724 nobody 18 0 435M 435M 435M S 0.0 43.1 0:00 httpd
20725 nobody 19 0 435M 435M 435M S 0.0 43.1 0:00 httpd
20726 nobody 19 0 435M 435M 435M S 0.0 43.1 0:00 httpd
-------------------------------------------
7:03pm up 2 days, 3:43, 1 user, load average: 0.45, 0.57, 0.78
79 processes: 76 sleeping, 3 running, 0 zombie, 0 stopped
CPU states: 0.0% user, 0.3% system, 0.0% nice, 99.6% idle
Mem: 1032132K av, 832776K used, 199356K free, 0K shrd, 18948K buff Swap: 2096376K av, 0K used, 2096376K free 615544K cached
PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND
767 nobody 15 0 1204 1204 1008 S 0.0 0.1 0:00 proftpd
903 nobody 17 0 2012 2012 1344 S 0.0 0.1 0:00 entropychat
908 nobody 15 0 872 872 592 S 0.0 0.0 0:00 melange
4504 nobody 15 0 320M 320M 320M S 0.0 31.8 0:00 httpd
4505 nobody 15 0 320M 320M 320M S 0.0 31.7 0:00 httpd
4506 nobody 15 0 320M 320M 320M S 0.0 31.8 0:00 httpd
4507 nobody 15 0 320M 320M 320M S 0.0 31.8 0:00 httpd
4508 nobody 15 0 320M 320M 320M S 0.0 31.8 0:00 httpd
4629 nobody 16 0 1860 1860 1404 S 0.0 0.1 0:00 proftpd
Any advice with this? How can i spot the problem on the server?
Regards!
sonic 12-06-2002, 03:07 PM 1. add more ram
2. remove web sites that use heavy cgi programs
Thanks for your reply, but the server is brand new as i said. I dont have any website/cgi program running on it. Thats the problem.
The problem is httpd probles are getting fatter and loads are now more than 20.2 :eek:
Regards!
sonic 12-06-2002, 03:26 PM do you have some heavy php/asp programs running on it , like a forum or chatroom?
or try this
/usr/sbin/lsof | grep PID
PID is the process id you can find when you execute "top"
Perfecthost 12-06-2002, 03:31 PM This is the strange part...you have no websites on the box, but an excessive amount of httpd processes. Have you restarted apache? Who set up the server?
Nope, i dont have any php/cgi programs. I dont have any script running at all. Thats the strange thing.
I dont have any website running on the server yet. Just the default apache webpage, this is because i havent move my sites to this new server. The big processes are the httpd ones.
I have cpanel with tomcat also installed, could this be the problem for the high loads?
I restarted httpd, this are the outputs from top now, though loads continue to be a little high for an empty server:
load average: 0.12, 0.17, 1.43
82 processes: 79 sleeping, 3 running, 0 zombie, 0 stopped
CPU states: 0.0% user, 0.1% system, 0.0% nice, 99.8% idle
Mem: 1032132K av, 805684K used, 226448K free, 0K shrd, 19064K buff
Swap: 2096376K av, 0K used, 2096376K free 620120K cached
1170 root 22 0 27584 26M 9372 S 0.0 2.6 0:02 /usr/local/jdk/bin/java -Djava.endorsed.dirs=/usr/local/ja
1180 root 15 0 27584 26M 9372 S 0.0 2.6 0:00 /usr/local/jdk/bin/java -Djava.endorsed.dirs=/usr/local/ja
1181 root 15 0 27584 26M 9372 S 0.0 2.6 0:00 /usr/local/jdk/bin/java -Djava.endorsed.dirs=/usr/local/ja
1182 root 15 0 27584 26M 9372 S 0.0 2.6 0:00 /usr/local/jdk/bin/java -Djava.endorsed.dirs=/usr/local/ja
1183 root 15 0 27584 26M 9372 S 0.0 2.6 0:00 /usr/local/jdk/bin/java -Djava.endorsed.dirs=/usr/local/ja
1184 root 15 0 27584 26M 9372 R 0.0 2.6 0:00 /usr/local/jdk/bin/java -Djava.endorsed.dirs=/usr/local/ja
1185 root 20 0 27584 26M 9372 S 0.0 2.6 0:00 /usr/local/jdk/bin/java -Djava.endorsed.dirs=/usr/local/ja
1186 root 16 0 27584 26M 9372 S 0.0 2.6 0:00 /usr/local/jdk/bin/java -Djava.endorsed.dirs=/usr/local/ja
1187 root 15 0 27584 26M 9372 S 0.0 2.6 0:00 /usr/local/jdk/bin/java -Djava.endorsed.dirs=/usr/local/ja
1196 root 15 0 27584 26M 9372 S 0.0 2.6 0:00 /usr/local/jdk/bin/java -Djava.endorsed.dirs=/usr/local/ja
1197 root 15 0 27584 26M 9372 S 0.0 2.6 0:00 /usr/local/jdk/bin/java -Djava.endorsed.dirs=/usr/local/ja
1198 root 15 0 27584 26M 9372 S 0.0 2.6 0:00 /usr/local/jdk/bin/java -Djava.endorsed.dirs=/usr/local/ja
1199 root 15 0 27584 26M 9372 S 0.0 2.6 0:00 /usr/local/jdk/bin/java -Djava.endorsed.dirs=/usr/local/ja
1200 root 15 0 27584 26M 9372 S 0.0 2.6 0:00 /usr/local/jdk/bin/java -Djava.endorsed.dirs=/usr/local/ja
1201 root 20 0 27584 26M 9372 S 0.0 2.6 0:00 /usr/local/jdk/bin/java -Djava.endorsed.dirs=/usr/local/ja
1202 root 20 0 27584 26M 9372 S 0.0 2.6 0:00 /usr/local/jdk/bin/java -Djava.endorsed.dirs=/usr/local/ja
1203 root 20 0 27584 26M 9372 S 0.0 2.6 0:00 /usr/local/jdk/bin/java -Djava.endorsed.dirs=/usr/local/ja
1204 root 16 0 27584 26M 9372 S 0.0 2.6 0:00 /usr/local/jdk/bin/java -Djava.endorsed.dirs=/usr/local/ja
1205 root 15 0 27584 26M 9372 S 0.0 2.6 0:00 /usr/local/jdk/bin/java -Djava.endorsed.dirs=/usr/local/ja
1374 cpanel 15 0 24372 23M 2268 S 0.0 2.3 0:00 interchange
1032 root 17 0 11176 10M 1620 S 0.0 1.0 0:00 /usr/bin/spamd -d
22329 nobody 16 0 7976 7976 7888 S 0.0 0.7 0:00 /usr/local/apache/bin/httpd -DSSL
22330 nobody 16 0 7976 7976 7888 S 0.0 0.7 0:00 /usr/local/apache/bin/httpd -DSSL
22331 nobody 16 0 7976 7976 7888 S 0.0 0.7 0:00 /usr/local/apache/bin/httpd -DSSL
22332 nobody 16 0 7976 7976 7888 S 0.0 0.7 0:00 /usr/local/apache/bin/httpd -DSSL
22333 nobody 16 0 7976 7976 7888 S 0.0 0.7 0:00 /usr/local/apache/bin/httpd -DSSL
21610 root 15 0 7968 7968 7880 R 0.0 0.7 0:00 /usr/local/apache/bin/httpd -DSSL
898 root 17 0 3820 3820 2304 S 0.0 0.3 0:00 cppop - accepting on port 110
943 root 15 0 3172 3172 2008 S 0.0 0.3 0:00 whostmgrd
Regards!
Hello Perfecthost, the server was setup by servint, its a fully managed server. i think they are checking out this issue, but i would like to help too on this because i need to have this server up asap.
Could this be a bug or something on cpanel/tomcat?
Regards!
mainarea 12-06-2002, 04:11 PM I'm no expert at this, but that shouldn't be happening :D. I'd suggest restarting your server, and if that doesn't fix it, maybe recompiling Apache.
- Matt
Perfecthost 12-06-2002, 04:50 PM Try re-booting your box. I just checked with a friend (also a host) and he encountered similar issues. He had TomCat installed with one of the later Red Hat versions. A reboot cleared it up. It seems some things were needing cleared out. It happened immediately after the TomCat installation.
However, I have a server running RH 6.2 and there were no problems after the installation of TomCat.
Good luck. I hear the folks at Servint know what they are doing and feel everything will be set right soon.
-Lamar
Well, apache was restarted, loads went down. httpd processes went down too, but now they are getting bigger again :) . At a slow rate but in a few minutes thety go from 10M to 16M
Maybe this is restart just delay the problem.
Ill try rebooting the server to see if that solves the problem. This is indeed RH 7.3, hope this problem get fixed after the reboot :(
xetok 12-06-2002, 05:50 PM Something's useing a large amount of RAM. Look at the SWAP used on your first posts before the reboot. Dipping into that much scratch will certainly give you high loads.
Are u sure it could be SWAP, isnt ok used meaning none of the swap being used?
Swap: 2096376K av, 0K used, 2096376K free 619844K cached
Though cached has 619, could this be part of the problem of the high loads?
My provider has not rebooted the server yet. httpd proc, are now 23M
5:07pm up 3 days, 1:46, 1 user, load average: 0.03, 0.03, 0.00
78 processes: 75 sleeping, 3 running, 0 zombie, 0 stopped
CPU states: 0.0% user, 0.1% system, 0.0% nice, 99.8% idle
Mem: 1032132K av, 808172K used, 223960K free, 0K shrd, 19076K buff
Swap: 2096376K av, 0K used, 2096376K free 620816K cached
PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND
24625 nobody 15 0 24536 23M 24216 S 0.0 2.3 0:00 /usr/local/apache/bin/httpd -DSSL
24626 nobody 15 0 24508 23M 24208 S 0.0 2.3 0:00 /usr/local/apache/bin/httpd -DSSL
24627 nobody 15 0 24508 23M 24208 S 0.0 2.3 0:00 /usr/local/apache/bin/httpd -DSSL
24628 nobody 15 0 24508 23M 24216 S 0.0 2.3 0:00 /usr/local/apache/bin/httpd -DSSL
24629 nobody 15 0 24508 23M 24216 S 0.0 2.3 0:00 /usr/local/apache/bin/httpd -DSSL
903 nobody 17 0 2012 2012 1344 S 0.0 0.1 0:00 entropychat
767 nobody 15 0 1204 1204 948 S 0.0 0.1 0:00 proftpd (accepting connections)
908 nobody 15 0 872 872 592 S 0.0 0.0 0:00 /usr/local/cpanel/bin/startmelange
Loads and cpu looks good but im still a litle bit concern of the process growing trough time...
xetok 12-06-2002, 06:12 PM nevermind, I'm an idiot. I was looking at the wrong thing :D
Output from errorlog. Im dont know much about Tomcat. Could someone shed some light about this kind of errors? Could this be the root of the problem causing the loads?
On my previous posts i can see on the SWAP data: 0k used
Did you mean MEM instead of SWAP.
[Fri Dec 06 22:10:19 2002] (debug) [jk_env.c (312)]: env.createBean2(): register channel.socket:localhost:8009 0x835c6e0
[Fri Dec 06 22:10:19 2002] (debug) [jk_env.c (248)]: env.createBean2(): Create [uri] /servlet/*
[Fri Dec 06 22:10:19 2002] (debug) [jk_env.c (383)]: env.getByName(): Get by name workerEnv 0x836de78
[Fri Dec 06 22:10:19 2002] (debug) [jk_env.c (312)]: env.createBean2(): register uri:/servlet/* 0x8361818
[Fri Dec 06 22:10:19 2002] (debug) [jk_env.c (248)]: env.createBean2(): Create [uri] /servlets/*
[Fri Dec 06 22:10:19 2002] (debug) [jk_env.c (383)]: env.getByName(): Get by name workerEnv 0x836de78
[Fri Dec 06 22:10:19 2002] (debug) [jk_env.c (312)]: env.createBean2(): register uri:/servlets/* 0x8362a80
[Fri Dec 06 22:10:19 2002] (debug) [jk_env.c (248)]: env.createBean2(): Create [uri] /*.jsp
[Fri Dec 06 22:10:19 2002] (debug) [jk_env.c (383)]: env.getByName(): Get by name workerEnv 0x836de78
[Fri Dec 06 22:10:19 2002] (debug) [jk_env.c (312)]: env.createBean2(): register uri:/*.jsp 0x8363dc0
[Fri Dec 06 22:10:19 2002] (debug) [jk_env.c (248)]: env.createBean2(): Create [threadMutex] (null)
[Fri Dec 06 22:10:19 2002] (debug) [jk_env.c (312)]: env.createBean2(): register threadMutex:5 0x8364920
[Fri Dec 06 22:10:19 2002] (debug) [jk_env.c (383)]: env.getByName(): Get by name workerEnv 0x836de78
[Fri Dec 06 22:10:19 2002] (error) [jk_logger_file.c (165)]: Can't open log file ${serverRoot}/logs/mod_jk.log
[Fri Dec 06 22:10:19 2002] (debug) [jk_env.c (248)]: env.createBean2(): Create [lb] lb
[Fri Dec 06 22:10:19 2002] (debug) [jk_env.c (248)]: env.createBean2(): Create [threadMutex] (null)
[Fri Dec 06 22:10:19 2002] (debug) [jk_env.c (312)]: env.createBean2(): register threadMutex:6 0x8366aa8
[Fri Dec 06 22:10:19 2002] (debug) [jk_env.c (248)]: env.createBean2(): Create [threadMutex] (null)
[Fri Dec 06 22:10:19 2002] (debug) [jk_env.c (312)]: env.createBean2(): register threadMutex:7 0x83672f8
[Fri Dec 06 22:10:19 2002] (debug) [jk_env.c (383)]: env.getByName(): Get by name workerEnv 0x836de78
[Fri Dec 06 22:10:19 2002] (debug) [jk_env.c (248)]: env.createBean2(): Create [threadMutex] (null)
[Fri Dec 06 22:10:19 2002] (debug) [jk_env.c (312)]: env.createBean2(): register threadMutex:8 0x8368b50
[Fri Dec 06 22:10:19 2002] (debug) [jk_env.c (312)]: env.createBean2(): register lb:lb 0x8365950
[Fri Dec 06 22:10:19 2002] (debug) [jk_env.c (248)]: env.createBean2(): Create [threadMutex] (null)
[Fri Dec 06 22:10:19 2002] (debug) [jk_env.c (312)]: env.createBean2(): register threadMutex:9 0x83693f8
[Fri Dec 06 22:10:19 2002] (debug) [jk_env.c (383)]: env.getByName(): Get by name ajp13:localhost:8009 0x835d7d0
[Fri Dec 06 22:10:19 2002] (debug) [jk_env.c (383)]: env.getByName(): Get by name ajp13:localhost:8009 0x835d7d0
[Fri Dec 06 22:10:19 2002] (debug) [jk_env.c (248)]: env.createBean2(): Create [handler.response]
[Fri Dec 06 22:10:19 2002] (debug) [jk_env.c (312)]: env.createBean2(): register handler.response: 0x8369c80
[Fri Dec 06 22:10:19 2002] (debug) [jk_env.c (248)]: env.createBean2(): Create [handler.logon]
[Fri Dec 06 22:10:19 2002] (debug) [jk_env.c (312)]: env.createBean2(): register handler.logon: 0x836a730
[Fri Dec 06 22:10:19 2002] (debug) [jk_env.c (248)]: env.createBean2(): Create [uri] /
[Fri Dec 06 22:10:19 2002] (debug) [jk_env.c (383)]: env.getByName(): Get by name workerEnv 0x836de78
[Fri Dec 06 22:10:19 2002] (debug) [jk_env.c (312)]: env.createBean2(): register uri:/ 0x836b868
[Fri Dec 06 22:10:19 2002] (debug) [jk_env.c (383)]: env.getByName(): Get by name status: 0x8358730
[Fri Dec 06 22:10:19 2002] (debug) [jk_env.c (383)]: env.getByName(): Get by name ajp13:localhost:8009 0x835d7d0
[Fri Dec 06 22:10:19 2002] (debug) [jk_env.c (383)]: env.getByName(): Get by name ajp13:localhost:8009 0x835d7d0
[Fri Dec 06 22:10:19 2002] (debug) [jk_env.c (383)]: env.getByName(): Get by name ajp13:localhost:8009 0x835d7d0
[Fri Dec 06 22:10:19 2002] ( info) [jk_workerEnv.c (403)]: workerEnv.init() ok /usr/local/apache//conf/workers2.properti
es
Failed loading /usr/local/Zend/lib/ZendOptimizer.so: /usr/local/Zend/lib/ZendOptimizer.so: cannot open shared object fil
e: No such file or directory
Warning: /boot/System.map has an incorrect kernel version.
Warning: /boot/System.map has an incorrect kernel version.
[Fri Dec 6 17:10:19 2002] [notice] Apache/1.3.27 (Unix) mod_jk2/2.0.0 mod_log_bytes/0.3 mod_bwlimited/1.0 PHP/4.2.3 Fron
tPage/5.0.2.2510 mod_ssl/2.8.11 OpenSSL/0.9.6b configured -- resuming normal operations
[Fri Dec 6 17:10:19 2002] [notice] suEXEC mechanism enabled (wrapper: /usr/local/apache/bin/suexec)
[Fri Dec 6 17:10:19 2002] [notice] Accept mutex: sysvsem (Default: sysvsem)
[Fri Dec 06 22:11:23 2002] (debug) [jk_env.c (157)]: env:Create child env 0x833a948 0x838ae28
[Fri Dec 06 22:11:25 2002] (debug) [jk_env.c (157)]: env:Create child env 0x833a948 0x838ae28
[Fri Dec 06 22:12:23 2002] (debug) [jk_env.c (157)]: env:Create child env 0x833a948 0x838ae28
[Fri Dec 06 22:13:23 2002] (debug) [jk_env.c (157)]: env:Create child env 0x833a948 0x838ae28
[Fri Dec 06 22:15:01 2002] (debug) [jk_env.c (157)]: env:Create child env 0x833a948 0x838ae28
Regards and thanks for your help! :)
|