Hi,
I have several extremely CGI-Intensive sites which generate quite an enormous amount of traffic. The site is an anonymous surfing website which hides the user's IP address, so it requires connecting to the other site and displaying the site for the user anonymously, and securely. Concequently my dedicated server uses a wealth of bandwidth, but more importantly an immense load on the server. Sometimes the load gets around 300, and just stays around that for a few hours until it settles down to about 100 where it usually averages. Other times it will grow significantly above 300 and cause the server to crash. This is a real nusience for me and my hosting company, because sometimes they need to get someone special in to fix the server due to corrupt system files, in the meantime my site loses quite a bit of traffic, and therefore money.

Right now the server is on a 1 GHZ Celeron Processor with 512 MB RAM. I can't afford all the spiffy hardware, since I only make a few hundred a month on the site with ads. I was concidering upgrading the RAM to 1024 or 1536 (if there are enough slots). I'm wondering from the expierienced how this will effect my performance. Would upgrading the CPU to Pentium 3 or 4 and/or to a higher megahertz improve it more? What would you recommend?

Thank you very much.
Jeff Dyer

a ram upgrade would probably help... but i think load balancing would be the best bet

1. Celeron is your problem.
2. 1024mb of RAM would REALLY help you.
3. If you are doing this for money, be like me and everyone else and just buy a quality server.

http://www.qsol.com/

Don't be stupid. It will pay off in the long run. Trust me. :)

The Celeron is hurting you (cache helps)
RAM always helps

Do buy a quality server, I can not vouch for the link that was posted, but SM makes good stuff.

kerplunk,

I've never heard about QSOL before. Please tell me, what is and how is your experience with them? They look interesting.

Thanks!

- krc

Greetings:

From working with Sun Microsystem engineers and other network engineers, we've found there is a correlation to bandwidth and the amount of RAM needed.

So the comments from everyone concerning moving to 1 GB of RAM are very appropriate.

Also, a dual processor P3 or P4 would be beneficial for the CGI use.

While load balancing would also help, in order to keep the number of machines load balanced to a minimum, I would recommend replacing the existing box with a dual P3/P4 (at least 1.26 GHz) and 1 GB of RAM.

Then look at load balancing.

A load of 300 on a single CPU machine? I'd be very carefull you don't fry the cpu before long.

More ram and even a duel CPU would realy help alot there. Still as long as the server stays up you should be ok.

did you optimize your PHP/PERL (or whatever you have) code you are running? Sometimes a badly coded script can cause high load etc....

ps. is your system swapping? check it out with 'free' command or 'top'.

Is this guy joking?

i.e. load averages: 300.00, 300.00, 300.00

? LOL... that is HORRIBLE.

This guy must be a cheapskate. If you have that much load, you need a better server. My server hosts A LOT... and the loads are NEVER over 1.00. (Unless I compile something.)

Anyway krc, qsol.com has a very reputable company. My host, DefenderHosting.com, has assured me of this. Although I have never bought a server from them, I was about to once, but my host built me a server for a little less money. Long story.

So... go to www.google.com and type in "qsol reviews" or something. I'm sure you will get nothing but good remarks and comments.

Ditch the Celeron. With those kinds of loads, I'm betting much of the work is highly repetitive. Some CPU caching would do wonders. You can get a PIII 1.4GHz w/512K cache for reasonable dollars and it might even work in the same board as the Celeron. Be sure to check on what CPU's your board will support before going this route.

Also, it could be that the server load is high because the CPU is tied up trying to swap to the hard drive. If your hard drive is IDE, that is most likely the case. Switch to a good SCSI card that takes most of the IO load off the CPU and put in a 10K RPM SCSI drive.

To get rid of swap altogether (or reduce it at least), try 1GB of RAM. Check your swap usage again after the upgrade. If you're still swapping, keep adding RAM until the swapping goes away. This will do wonders for your performance and provide much better use of the CPU. You might even be able to avoid the SCSI system if you're not swapping. RAM is dirt cheap. You can put in 2GB of PC2100 ECC Registered DDR RAM for under $500 (provided your board can handle it).

Of course, this doesn't solve your cash flow problem. Consider creating a second ad-free version of the web site and all users who donate a dollar or more via PayPal get the ad-free version of the site. Those visitors will use less of your resources btw because they won't be chewing up your CPU with a banner serving CGI.

kerplunk:

Thanks for the link. Glad to hear that others have heard they are a good company too.

Thanks for your replies.

I'll contact my host and see what I can do to replace my Celeron with a Pentium and higher megahertz.

Also, the script I'm using wasn't programmed by myself, I imagine the script is programmed really poorly which causes the load to increase so much, but there's no way around that right now.

kerplunk, why don't you close your mouth. You don't even know me so why on earth would you call me a cheapskate, I don't have a lot of money to go around since I'm still in high school right now.

I'm not interested in purchasing a new server. Thanks anyway. The server doesn't seem to be using too much swap, at most it uses 20-30 megs of the 1024.

Thanks,
Jeff Dyer

I'm 16 and a junior in high school. I wasn't attacking you. I am sorry if you thought that.

The only thing that I was trying to get out here is that: Buy the dreammachine now. Get a loan from the bank or something. (Interest rates are very good.) Then, just pay it back monthly... or why even try and run a website?

I don't know what I am trying to say here, now... but... you posted this topic, if you thought you were not going to get any replies like mine, then you are/were dreaming. :)

Cheer up, spend some money, and have a great day!

1024

Way too much swap space - a gaming server pushing 11MBPS I saw used 127MB of swap at times..... I only ever now build servers and allocate 128MB to swap, it would be such a waste to do otherwise :D

Please run top and give us the top set of lines. There is a lot of speculation as to what your problem is, but giving us this information we will be able to give you some more valuabe information.

I think it's been decided that he should upgrade the machine, both CPU and memory and he'll be fine most probably, but yes post those top lines to "ps aux" :D

Kerplunk,
that's strange, you're 16 and in juniour high still? I'm 16 and in the senior year of high school. Strange stuff.

I have more than enough money to buy the machine, it's not the problem I want to try to avoid spending that much money right now however, I don't even believe there is any place to co-locate where I live.

By the way, I expected replies like yours, I don't mind people saying:
"Is this guy joking?

i.e. load averages: 300.00, 300.00, 300.00

? LOL... that is HORRIBLE. "...however, after that since you made a comment directed to me which has nothing to do with my question, or the solution and really served no purpose other than an attempt to annoy me. Anyway let's not make a big deal out of it.

I'll post the 'top' and 'ps aux' info when the server starts to get loaded, right now the server load is only around 20 so it's not worth posting.

By the way, I didn't choose that much swap, my dedicated server automatically allocated twice the amount of ram. It shouldn't hurt it though, it's not like my site has a lot of large files it needs hard drive space for.

The MaxClients in apache was set to 256, so I set it down to MaxClients 150, in attempt to limit connection and keep the server less busy. Will this help? Should I make it lower, and is there any other settings in apache that would help improve performance?

Thanks,
Jeff Dyer

Here's the top part of the "top" command:
11:04am up 23:26, 2 users, load average: 106.69, 103.45, 69.55
340 processes: 243 sleeping, 83 running, 14 zombie, 0 stopped
CPU states: 83.1% user, 16.8% system, 0.0% nice, 0.0% idle
Mem: 513184K av, 418408K used, 94776K free, 0K shrd, 10320K buff
Swap: 1052216K av, 0K used, 1052216K free 203600K cached

Here's the top part of the "ps aux" command (I don't know if you need more):
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 1 0.0 0.0 1372 480 ? S Sep06 0:03 init [3]
root 2 0.0 0.0 0 0 ? SW Sep06 0:00 [keventd]
root 3 0.0 0.0 0 0 ? RWN Sep06 0:00 [ksoftirqd_CPU0]
root 4 0.0 0.0 0 0 ? SW Sep06 0:00 [kswapd]
root 5 0.0 0.0 0 0 ? SW Sep06 0:00 [bdflush]
root 6 0.0 0.0 0 0 ? SW Sep06 0:00 [kupdated]
root 7 0.0 0.0 0 0 ? SW Sep06 0:00 [mdrecoveryd]
root 81 0.0 0.0 0 0 ? SW Sep06 0:00 [khubd]
root 606 0.0 0.1 1428 560 ? S Sep06 0:00 syslogd -m 0
root 611 0.0 0.0 1364 444 ? S Sep06 0:00 klogd -x
rpc 631 0.0 0.1 1512 552 ? S Sep06 0:00 portmap
rpcuser 659 0.0 0.1 1560 720 ? S Sep06 0:00 rpc.statd
root 808 0.0 0.2 2620 1232 ? S Sep06 0:00 /usr/sbin/sshd
root 842 0.0 0.1 2196 924 ? S Sep06 0:00 xinetd -stayalive

Here is your problem right here

CPU states: 83.1% user, 16.8% system, 0.0% nice, 0.0% idle

Your cpu is hammered. It does not appear that you are in that big of a memory crunch...not that it wouldn't help, but your bigger problem is cpu. Can you post your top a few more times....I don't really see anything in the list that shows what process(es) are using your cpu.

I believe 300 load averages had everything to do with your question, and incase you haven't figured this out yet:

Your server needs an upgrade. (Or maybe you need a new server.)

* stops wasting his time on this post *

you need to tune your scripts... if it's perl, consider moving to php or c cgi's....

I found out limiting the MaxClients to 128 really helped lower the load, and my hits aren't any lower...not sure what that's all about but if it works, it works.

8:35pm up 2:57, 1 user, load average: 27.16, 26.36, 20.11
148 processes: 118 sleeping, 19 running, 11 zombie, 0 stopped
CPU states: 77.3% user, 13.5% system, 0.0% nice, 9.1% idle
Mem: 513184K av, 100216K used, 412968K free, 0K shrd, 5732K buff
Swap: 1052216K av, 0K used, 1052216K free 27820K cached
.....five minutes later.....
8:40pm up 3:02, 1 user, load average: 15.00, 19.85, 19.09
154 processes: 114 sleeping, 30 running, 10 zombie, 0 stopped
CPU states: 82.9% user, 17.0% system, 0.0% nice, 0.0% idle
Mem: 513184K av, 111128K used, 402056K free, 0K shrd, 5732K buff
Swap: 1052216K av, 0K used, 1052216K free 27844K cached

dandanfirema, The most I can upgrade my CPU to is P3 1.2 Ghz. Do you think that would help at all? If it'll help I'll go for it.

Jeff

What we really need is the part below what you have posted above. It should list what processes are running at the time and how much cpu and memory they take. The upgrade from Celeron to P3 1.2 should help, but Lets take a look at the processes and see what they are.

Here's the entire thing:

8:53pm up 3:15, 1 user, load average: 24.20, 23.46, 22.00
172 processes: 123 sleeping, 45 running, 4 zombie, 0 stopped
CPU states: 83.3% user, 16.6% system, 0.0% nice, 0.0% idle
Mem: 513184K av, 120564K used, 392620K free, 0K shrd, 5732K buff
Swap: 1052216K av, 0K used, 1052216K free 27860K cached

PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND
11241 nobody 16 0 2248 2248 1312 R 4.2 0.4 0:00 nph-go.cgi
11232 nobody 16 0 2252 2252 1312 R 3.8 0.4 0:00 nph-go.cgi
11236 nobody 16 0 2244 2244 1308 R 3.8 0.4 0:00 nph-go.cgi
11240 nobody 16 0 0 0 0 Z 3.8 0.0 0:00 nph-p12.cgi <defunct>
11251 nobody 16 0 2252 2252 1312 R 3.8 0.4 0:00 nph-go.cgi
11234 nobody 16 0 2244 2244 1308 R 3.4 0.4 0:00 nph-go.cgi
11237 nobody 16 0 2268 2268 1324 R 3.4 0.4 0:00 nph-go.cgi
11238 nobody 16 0 2252 2252 1312 R 3.4 0.4 0:00 nph-go.cgi
11248 nobody 16 0 2260 2260 1320 R 3.4 0.4 0:00 nph-go.cgi
11250 nobody 16 0 1880 1880 1104 R 3.4 0.3 0:00 nph-p12.cgi
11252 nobody 16 0 2012 2012 1108 R 3.4 0.3 0:00 nph-go.cgi
11245 nobody 16 0 1880 1880 1104 R 3.0 0.3 0:00 nph-p12.cgi
11231 nobody 16 0 2268 2268 1324 R 2.3 0.4 0:00 nph-go.cgi
11244 nobody 16 0 1616 1616 1060 R 2.3 0.3 0:00 nph-go.cgi
11255 nobody 16 0 1604 1604 1060 R 2.3 0.3 0:00 nph-go2.cgi
11257 nobody 16 0 1792 1792 1104 R 2.3 0.3 0:00 nph-go.cgi
11246 nobody 16 0 1600 1600 1060 R 1.9 0.3 0:00 nph-p12.cgi
11253 nobody 16 0 1572 1572 1060 R 1.9 0.3 0:00 nph-p12.cgi
12392 root 15 0 1140 1140 836 R 1.5 0.2 0:21 top
11223 nobody 16 0 2248 2248 1312 R 1.5 0.4 0:00 nph-go.cgi
11227 nobody 16 0 2260 2260 1320 R 1.5 0.4 0:00 nph-go.cgi
11215 nobody 16 0 2268 2268 1324 R 1.1 0.4 0:00 nph-go.cgi
11247 nobody 16 0 1324 1324 1008 R 1.1 0.2 0:00 nph-go.cgi
11221 nobody 16 0 2252 2252 1312 R 0.7 0.4 0:00 nph-go.cgi
11226 nobody 16 0 2252 2252 1312 R 0.7 0.4 0:00 nph-go.cgi
11258 nobody 16 0 1256 1256 976 R 0.7 0.2 0:00 nph-go.cgi
23475 nobody 15 0 2168 2168 2112 S 0.3 0.4 0:02 httpd
23515 nobody 15 0 2168 2168 1972 S 0.3 0.4 0:02 httpd
11225 nobody 16 0 2252 2252 1312 R 0.3 0.4 0:00 nph-go.cgi
1 root 15 0 480 480 420 S 0.0 0.0 0:05 init
2 root 15 0 0 0 0 SW 0.0 0.0 0:00 keventd
3 root 34 19 0 0 0 RWN 0.0 0.0 0:00 ksoftirqd_CPU0
4 root 15 0 0 0 0 SW 0.0 0.0 0:00 kswapd
5 root 25 0 0 0 0 SW 0.0 0.0 0:00 bdflush
6 root 15 0 0 0 0 SW 0.0 0.0 0:00 kupdated
7 root 25 0 0 0 0 SW 0.0 0.0 0:00 mdrecoveryd
81 root 15 0 0 0 0 SW 0.0 0.0 0:00 khubd
599 root 15 0 560 560 472 S 0.0 0.1 0:01 syslogd
604 root 15 0 444 444 384 S 0.0 0.0 0:00 klogd
624 rpc 15 0 552 552 468 S 0.0 0.1 0:00 portmap
652 rpcuser 19 0 720 720 624 S 0.0 0.1 0:00 rpc.statd
801 root 15 0 1232 1232 1112 S 0.0 0.2 0:00 sshd
835 root 15 0 924 924 732 S 0.0 0.1 0:00 xinetd
876 root 15 0 1816 1816 1300 S 0.0 0.3 0:00 sendmail
896 root 19 0 440 440 384 S 0.0 0.0 0:00 gpm
914 root 15 0 620 620 540 S 0.0 0.1 0:00 crond
966 daemon 15 0 524 524 460 S 0.0 0.1 0:00 atd
1017 root 15 0 1916 1916 1908 S 0.0 0.3 0:00 httpd
1073 root 15 0 588 588 520 S 0.0 0.1 0:00 bwbar
1147 root 15 0 3356 3356 1472 S 0.0 0.6 0:00 miniserv.pl
1174 root 15 0 3860 3860 1536 S 0.0 0.7 0:00 miniserv.pl
1181 root 16 0 400 400 344 S 0.0 0.0 0:00 mingetty
1182 root 16 0 400 400 344 S 0.0 0.0 0:00 mingetty
1183 root 16 0 400 400 344 S 0.0 0.0 0:00 mingetty
1184 root 16 0 400 400 344 S 0.0 0.0 0:00 mingetty


Regards,
Jeff

Any idea what those 2 cgi files are?

All the cgi's that start with nph- are the proxies.

Just a matter of too little cpu power it appears.

That's strange because my host said that when the server crashed it was a case of not having enough ram. However, since I lowered MaxClients that doesn't seem to be a problem anymore.

Jeff Dyer

When you run out of ram, I would expect to see your server start swapping. If this were the case, the line that says

0K used

would actually have a number....a big one. Now, no one knows what it looked like before it crashed, but atleast right now that does not seem to be the problem