Hello everyone, on on of my RAQ 4's I have noticed a tremendous problem over the past few days, it has been freezing several times a day, as it overloads it's ram. I have been unable to deduce what the problem is and am hoping that perhaps someone else could give me some pointers as to how to fix it.

First off, my RAQ has an average of 3-4 megs of ram free after running for ten minutes. This is very odd, as it used to have 30+ on average. I have been running 'top' and the top 10 processes use only 10% of the ram, though the system claims that there are over 70 processes sleeping, with 2-4 active. My traffic is quite low on the raq, and I've noticed it tends to freeze every 5 hours or so, as a result of the ram being overloaded. The dilemma is locating what is affecting the ram. I was told it might be the httpd.conf file changes I made, as the server always seems to work ok again after resetting httpd (at least for a short while)
These are the changes I've made:
<VirtualHost xxx.xx.xxx.xx> (ip changed for security reasons)
ServerName www.frst.net
ServerAdmin admin
DocumentRoot /home/sites/site25/web
ServerAlias frst.net
RewriteEngine on
RewriteCond %{HTTP_HOST} !^216.40.195.12(:80)?$
RewriteCond %{HTTP_HOST} !^www.frst.net(:80)?$
#RewriteRule ^/(.*) http://www.frst.net/$1 [L,R]

#inserted this crap
ErrorDocument 404 http://www.frst.net/cgi-bin/redirect.cgi
ServerAlias *.frst.net
RewriteMap lowercase int:tolower
RewriteEngine on

RewriteCond ${lowercase:%{HTTP_HOST}} !^$
RewriteCond ${lowercase:%{HTTP_HOST}} !^www\.frst\.net$

RewriteCond ${lowercase:%{HTTP_HOST}} ^(www\.|)([^.]+)\.frst\.net$
RewriteRule ^(.+) ${lowercase:%{HTTP_HOST}}$1 [C]
RewriteRule ^(www\.|)([^.]+)\.frst\.net(.*) /f/$2$3 [L]

RewriteCond ${lowercase:%{HTTP_HOST}} ^(www\.|)[^.]+\.[^.]+\.frst\.net$
RewriteRule ^(.+) ${lowercase:%{HTTP_HOST}}$1 [C]
RewriteRule ^(www\.|)([^.]+)\.([^.]+)\.frst\.net(.*) /f/$3/$2$4 [L]
#end of insert

RewriteOptions inherit
AliasMatch ^/~([^/]+)(/(.*))? /home/sites/site25/users/$1/web/$3
AddType application/x-httpd-php .php4
AddType application/x-httpd-php .php
# AddHandler chiliasp .asp
# AddHandler chiliasp .asa
# Start of mod_rewrite configuration
RewriteMap lowercase int:tolower
RewriteEngine on

RewriteCond ${lowercase:%{HTTP_HOST}} !^$
RewriteCond ${lowercase:%{HTTP_HOST}} !^www\.frst.net$

RewriteCond ${lowercase:%{HTTP_HOST}} ^([^.]+)\.frst\.net$
RewriteCond /home/sites/site25/web/%1 -d
RewriteRule ^(.+) ${lowercase:%{HTTP_HOST}}$1 [C]
RewriteRule ^([^.]+)\.frst\.net(.*) /f/$1$2 [L]
# End o' snippet

AddHandler cgi-wrapper .cgi
AddHandler cgi-wrapper .pl
AddHandler server-parsed .shtml
AddType text/html .shtml
</VirtualHost>

Anything look wrong?

Thanks guys.

hmm, i know i won't be much help, but how much RAM are you running on the machine? perhaps upping it would help. linux loves RAM and so do RAqs.

maybe you could post what your TOP looks like, so someone could check it out.

also, if you think it's an apache problem, be sure to check in your httpd.conf as well for:

HostnameLookups
on or off. i've seen it recommended that you turn this off when performance tuning apache.

MaxSpareServers
apache default is 10 i believe. this is the setting for the amount of spare idle child server processes the box will have.

MinSpareServers
default is 5 i think. minimum amount of spare idle child server processes before apache spawns more.


hmm. there are others, perhaps you should search around for apache tweaks.

I don't know what you have loading it up, but one way to tell is to one day while you're working at your workstation, log into your server, su to root (tell ya why in a minute, same pass as admin most likely), start up top (type top in a terminal), and let it run in the background until you notice your server starting to load up.

At this point, you will be able to see what's taking up all your cpu, while in top, hit shift-m .. this will sort by memory usage and you will be able to see what's eating up all the memory. Take note of the process id (number all the way to the left in top), control-c out of top, and kill that process. Since you're already root, you won't waste time by switching to root now that the server's loaded up.. many a time i've been admin and didn't have the time to su to root and kill a runaway process before the server died.. heh. Be careful though, if it's mysql, postgres, or other db that's having a problem, you will probably want to shut it down with the init script (/etc/rc.d/init.d/mysql stop) instead of killing it in order to shut it down without corrupting the db.

You say you have a raq4, are you using asp? I've seen some asp coding problems that will do what you're explaining.. i'm not saying that you have bad asp coding of course.. just something to check :)

Also, the server freezing after loading up may have something to do with the latest kernel patch from cobalt (if you have it installed). I had to back it out because every morning when logrotate would run, the memory would load up and the server would not be able to load anything. Check your kernel log (/var/log/kernel) for messages like 'unable to load library...' right before the time of when it dies. With this problem, the server stays up and running, and you can ping it, but you can't get in via any of the open ports. Httpd, ftpd, telnetd, sshd, etc.. don't respond.

Anyway, I hope this helps you to figure out your problem.

Good luck! :)

I sorted by Shift-M and found 17-25 httpd processes running and eating up almost all of the ram. Is this normal, and if not how should I stop it?
Thanks.

its normal that there are processes running as far as i know, but it might help if you tell how much ram is on the raq.