Hey All,
My server has been crashing steadily, around 3 times a day right now... I usually get CPU usage of 50% or more every time it crashes...

My server is a Cobalt Raq 3 with 128 RAM. I am running a large MySQL database (40,000+ rows) and my entire site is dynamic using PHP. I am currently receiving 3 million page views a month.

Is it conceivable that this is just a problem with my hardware? Or do I just have to fine-tune my server?
If it is hardware, to what system would I have to update?

Thanks,
Jeremy Ferwerda
www.mxtabs.net

Raqs have a tendancy to crash when under moderate load. I don't know if it is a hardware issue (insufficient cooling?) or just a Redhat issue.

Considering that you're using a large SQL database, you'd probably see much better performance, lower load, and likely more stability, if you added another 128MB of RAM.

I looked in my error logs, and saw this message :

handler "cgi-wrapper" not found for: (a cgi file)
My error log is fairly large (1 meg), and full of these messages from the 23rd and 24th.

Would this affect performance? How can I fix this error?

That shouldn't affect performance, you're probably luck it isn't loading if you're having problems with load already heh. Check your apache configuration file to see if cgi is setup correctly.

Although we did not have crashes, we had similar problems running a site on a Redhat box with a PII300, IDE drives, and 128MB of RAM. Our mysql database had grown quite large and was starting to take up more RAM. As a result, the box had to use swap space a lot more which nailed performance. We started getting application timeout errors and such. When we upped the Apache spareservers configuration, it only made manners worse because now Apache was taking more memory. We added another 256MB of ram and our problems were reduced.

Some of the problems came from poor SQL queries which were re-written. We also used mod_DBI (we're using PERL) to maintain persistent database connections. This really helped as well as some tweaks to the database (new indecies and optimizing the tables). If your running MySQL, you should run the OPTMIZE TABLE command if you have made any signifanct additions/deletions/modifications to the table. This will bring the indecies back into sync and can have a major speed boost.

The server is crashing as I am writing this :

I am in the top command : 73% CPU usage
303 processes: 302 sleeping
83% idle processes
mem: 2508k free

What does this indicate?

I would say your server is ****y with no resources at all 3millions hits with PHP and SQL just on 128ram?
I don't think it can hold that much.

Originally posted by mxtabs
The server is crashing as I am writing this :

I am in the top command : 73% CPU usage
303 processes: 302 sleeping
83% idle processes
mem: 2508k free

What does this indicate?

How much swap is in use? If you're using a lot of swap, it's time to get more ram.

I still have a lot of swap free

I'm not sure if I should upgrade my RAM, or move off the RAQ and onto a better Linux server.... what do you guys suggest?

Possible network hicups (the most pupular exuse for NOCs) or a Bug.What did your host had to say about this?

I asked my host to upgrade me to 256 RAM... but this morning when I checked the 'top' command.... all my new memory was already used up.

Could this be a memory leak?

Why do you have so many processes running? Is it possible that a script is not dying off as it should or is your site just that busy?

You can use
ps -axf
to list all of your procs in a tree format. This will show you which programs are running and what the child/parent relationships are.

Linux and RAM
Linux will almost always use all of your RAM regardless of what processes are running or how much RAM you have. For example, our development box has 384MB of RAM and is only used by about 3-5 people at a time. It runs Coldfusion Server, Apache, MySQL and a couple of other tools. The load is pretty light, but there is never more than 13MB of RAM free. This is because Linux uses any available RAM for file cache and other purposes. As long as your swap is mostly (>90%) free then you should be ok.

I used the ps option that you gave, and it seems that most of the processes are httpd.... they all say httpd.conf

Previously I had been using -aux, and this showed the same thing.

I am not sure why there are so many...

Those should be fine....there should be one for every Server that is running according to your settings in httpd.conf. Is this a high volume site? It could be that your spare httpd servers are not dying off. Can you do a quick restart of apache?


Have you looked in your server's log files for any clues?

/var/log/messages
should have some info in it.
you can use:
tail messages
to spit out the last lines of the file --

This is a high volume site, I receive an average of about 110,000 page views a day.

I ran 'tail messages' and found 2 isntancs where someone has tried to log in FTP as 'anonymous' Interested, I did a grep, and piped it into a wordcount to see how many times this has happened.
It returned '1612'

Not sure if this is a problem... probably isnt

No other abnormal messages