Web Hosting Talk







View Full Version : Apache Memory Issues


bert
09-07-2002, 11:13 AM
I wonder if someone has even experienced this. On one of our servers, Apache eats up memory as hell. It uses all available memory and eventually starts eating swap memory until the server crashes. The process is quite slow, it might take about 5 days for Apache to use all available swap and if we restart apache, the swap will then be freed again and the process will start all over. We thought this would be hardware, but we replaced the entire memory on the server and that did not make a difference.

The server is a standard RedHat 7.2 install / 2.4.18 #2 SMP with Cpanel/WHM, 2 PIII 1GHz processors, 1 Gig of RAM and dual SCSI drives. This is the exact same hardware and software configuration we have on 5 other servers that do not have this problem.

I have seen that apache processes use in excess of 200 Megs of ram and the longer apache runs, the bigger the processes get.

We don't want to add customers to this server unless it becomes stable, but we've tried everything and nothing appears to work.

Any thoughts?

MultiVol
09-07-2002, 11:30 AM
Dump it and get another server and reconfig it?

bert
09-07-2002, 11:34 AM
Hehe, that is the easy solution :D

MultiVol
09-07-2002, 12:05 PM
lol :D

Anyways, hope you find out why.

2host.com
09-08-2002, 08:13 PM
Originally posted by bert
I wonder if someone has even experienced this. On one of our servers, Apache eats up memory as hell. It uses all available memory and eventually starts eating swap memory until the server crashes. The process is quite slow, it might take about 5 days for Apache to use all available swap and if we restart apache, the swap will then be freed again and the process will start all over. We thought this would be hardware, but we replaced the entire memory on the server and that did not make a difference.

The server is a standard RedHat 7.2 install / 2.4.18 #2 SMP with Cpanel/WHM, 2 PIII 1GHz processors, 1 Gig of RAM and dual SCSI drives. This is the exact same hardware and software configuration we have on 5 other servers that do not have this problem.

I have seen that apache processes use in excess of 200 Megs of ram and the longer apache runs, the bigger the processes get.

We don't want to add customers to this server unless it becomes stable, but we've tried everything and nothing appears to work.

Any thoughts?


Were you able to run any commands while this was happening? What do your Apache logs show before the crash or while it's happening? How about top? ps? netstat? lsof? tcpdump? Anything to show you the processes, what is consuming it, how many there are, what connections are being made to what and how many, and so on? Also check your messages file to look for any vm errors and the like before the crash. Also check dmesg while, before or after it happened, as well as after a reboot too if that's the case. It shouldn't be too difficult to track down the cause.

bert
09-08-2002, 08:21 PM
Hi Robert,

Thanks, but I have tried all of the above with no success. The problem is that the processes show as /usr/local/apache/bin/httpd -DSSL and as user nobody.

I am now starting to believe that this has to do with mod_gzip. It is really the only thing that I have not tested. I just disabled it, but I guess I will have to wait a couple of days to see if the process repeats.

Thanks for your help :)

2host.com
09-08-2002, 08:27 PM
Originally posted by bert
Hi Robert,

Thanks, but I have tried all of the above with no success. The problem is that the processes show as /usr/local/apache/bin/httpd -DSSL and as user nobody.

I am now starting to believe that this has to do with mod_gzip. It is really the only thing that I have not tested. I just disabled it, but I guess I will have to wait a couple of days to see if the process repeats.

Thanks for your help :)

So then it does show it as Apache in the processes, but do you see how many processes are running and how much each is taking for memory and CPU? You can just enable server-status and view that URL page when you see the log getting too high nd you can match up the memory hog process ID with the PID in the server-status page to see what script or file is causing the load, and on what domain (along with the web path to the file). You can then determine the cause. I'd suggest that you implement some limits in your configuration though. It's best to prevent things like this from happening, rather than trying to deal with a rouge CGI or PHP script, etc., because you can't predict when a client will upload a poorly coded script.

bert
09-08-2002, 08:34 PM
That is the problem, every single apache process shows 300 or more megs of memory usage at all times, you can't track it to a particular domain because on this machine there are only about 10 to 12 domains and they are not too active. It is very weird indeed. When I match up the memory hog process ID with the PID in the server-status page, it shoes different domains every time.

2host.com
09-08-2002, 08:45 PM
Next time it happens, paste in this thread showing the results of a "ps aux" and maybe a "ps auxww". Then paste in the output of server-status. Also include the output of netstat -an, and it might help if you get another set of eyes looking at the problem. I'd suggest you also check the size of any work files that gzip might have created. Perhaps some site has some huge file that's actually a binary that mod_gzip is trying to compress because it's named with a file extension that it thinks is text. If that'd the case you can have it check the file type, rather then only by extension for compression. That's caused some people problems by not doing it. Rather than continue to guess, hopefully you won't have the problem, but I'm sure you'd like to know what the cause is anyway, so post here (or email me) that information next time it happens and I'll see if I can suggest anything.

bert
09-08-2002, 09:18 PM
Perhaps some site has some huge file that's actually a binary that mod_gzip is trying to compress because it's named with a file extension that it thinks is text. If that'd the case you can have it check the file type, rather then only by extension for compression. That's caused some people problems by not doing it.


Thanks again Robert. I think this is the cause. I have disabled mod_gzip just to see if that helps. If it does, I will then enable it again and will closely monitor the wrk files. I will paste output here if the processes start to eat up memory again.