I haven't really done anything new other then added another forum on the server (Invision).
Basically the problem is the random spikes I get. The main forum that is running on the server haven't really increased in terms of sim connections. Its usually around 180 - 220 during peak times.
On TOP there is usually around 250 tasks. When this thing started happening it would spikes to 500+ tasks which totally kills the server. Its pretty random also. Sometimes it'll go fine for a few hours and then suddenly climbed slowly up and then the server dies. For now its going to crash whenever I let apache run for an hour or so.
The only way I can restore is to restart apache. It has come to the point where I've had to put in restart apache in the cronjob list. Restart it every few minutes since I'm not always here watching over it. This is obviously a shortterm solution to keep the forum up while I troubleshoot this.
I still don't get why you think it's something else...
Simple math and looking at your top output should tell you the system is swapping itself to death... ie. it's ran out of physical memory... all the swapping is causing your IO wait to skyrocket... causing all your processes to run sluggish... it's a downward spiral once your system starts actively swapping.
sar -rq 1 999999 > watchmecrash.txt&
then look at the output of the file once the system starts slowing down...