Web Hosting Talk : Web Hosting Main Forums : Hosting Security and Technology : Apache and php issue - very slow responds

[bluemaster]

Hi,
I have some problems with my current dedicated server and I need your advices about optimising or upgrading hardware etc. I will try to summarize the problems with the past and now.

My websites run on a server with Core2Duo 2,53ghz cpu and 4GB ram now. WHM 11.23.2 cPanel installed on Apache/2.0.63 and PHP/5.2.5..

During the day for about 2 hours it gets high traffic than other hours of the day. I think it gets around 2000-4000 online people at that times may be more or less. And all of the my websites become slower. Event not responding sometimes.. It tooks many time to loading. And sometimes it fails to load only half of the page comes. And sometimes it gives error about unable to reach the page etc.

I optimized mysql, added more indexes and changed most of the php mysql queries to get more performance. It help me to lower mysql CPU usage.. It was around %100-%160.. Now it is around %60 at that bussy times. But still I had responding problem. Still at the bussy times I cant reach my websites.

The interesting thing is that the websites was on a vps before this dedicated. With the same traffic.. And that vps was only had 1gb ram. And it was no worse than this.. I didnt get any performance improvement by changing the vps to this dedicated..

Another point is that;
Now when I check the status with the top command I see that used memory is over 3GB.. At the VPS top command gave aroun 600-800mb
so on...

Now I dunno what to do for further improvement. I think I did my best for mysql and php..

I read something about eaccelerator and zend optimizer for increasing the speed of php. What should I do? Even html based websites not responding at the bussy times. So php accelators will help me?

Any help will be very appreciated..
Please give me some suggestion what should I do..

Regards..

[Alex_M]

Without knowing a specific of your current server installation and without seeing some CPU/RAM usage logs it's difficult to say what causes such slowness. If you don't see any improvements in comparison with your old VPS, that means your current DED server software is configured in different way (not sure if there's worse hardware if you say that it has four times more RAM). What's recommended is software optimization:

- issue force Cpanel update (/script/upcp --force) and try to rebuild Apache with necessary modules from the beginning (/scripts/easyapache);
- open "httpd.conf" and pay attention on these Apache settings:

MaxKeepAliveRequests xxx
KeepAliveTimeout xxx
MaxClients xxx
MaxRequestsPerChild xxx

(you may want to reduce some of them)

- open "/etc/my.cnf" and specify some important global variables there, like:

set-variable = max_connections=xxx
set-variable = max_user_connections=xxx
set-variable = max_allowed_packet=xxxM

and restart MySQL then.
As I have said, I can't advice you some certain variables values there because I don't know how many websites, database, visits etc. you have on your DED-server, what your current settings are and so on.
Just try to reduce current MySQL/Apache variables values.

And as I know, there are no in-built accelerators in Cpanel but you do can compile ZendOptimizer (when running "easyapache" script).
It's also recommended you define "who and what" causes the main part of RAM/CPU load. You can do it via "top" and "strace" commands or by looking at corresponding statistic at WHM panel.

[bluemaster]

Thanks so much.
The server is so fresh. Apache and cpanel installed 4 days ago (as i remember)
And support tech of the server company said there is no hardware issue. For them there is no problem at server and the problem is my websites. But I cant understand that even a vps can cope with this website why I dont get more performance on this dedicated server Btw, the site gets around 40-50k daily visitors. I have few small websites in that dedicated too but they dont get much visitors.

Here the "top" results from ssh. Now it is not bussy. Four hours later the website will become so bussy about the traffic. As you see used ram is not 3gb right now because of low traffic.

top - 15:16:28 up 1 day, 18:19,  3 users,  load average: 0.16, 0.22, 0.25
Tasks: 251 total,   1 running, 244 sleeping,   0 stopped,   6 zombie
Cpu(s): 14.1% us,  3.8% sy,  0.0% ni, 81.6% id,  0.0% wa,  0.2% hi,  0.3% si
Mem:   4031480k total,  2789340k used,  1242140k free,   379464k buffers
Swap:  4192924k total,        0k used,  4192924k free,  1139928k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND            
15010 mysql     15   0  257m  38m 4068 S 18.3  1.0  19:04.17 mysqld             
22431 nobody    15   0 96380 7504 3412 S  8.0  0.2   0:00.29 httpd              
22704 nobody    15   0 96032 6936 3172 S  2.0  0.2   0:00.39 httpd              
21970 nobody    16   0 96372 7496 3412 S  0.7  0.2   0:00.37 httpd              
21964 nobody    15   0 48448 6464 2856 S  0.3  0.2   0:00.17 httpd              
22147 nobody    16   0 96408 7620 3420 S  0.3  0.2   0:00.50 httpd              
22622 nobody    15   0 96096 7036 3260 S  0.3  0.2   0:00.11 httpd              
22675 nobody    15   0 96364 7424 3344 S  0.3  0.2   0:00.21 httpd              
22688 nobody    15   0 48448 6436 2848 S  0.3  0.2   0:00.01 httpd              
22698 nobody    15   0 48448 6392 2812 S  0.3  0.2   0:00.01 httpd              
22729 nobody    16   0 48448 6424 2840 S  0.3  0.2   0:00.01 httpd              
    1 root      16   0  4772  568  468 S  0.0  0.0   0:01.01 init               
    2 root      RT   0     0    0    0 S  0.0  0.0   0:00.05 migration/0        
    3 root      34  19     0    0    0 S  0.0  0.0   0:00.10 ksoftirqd/0        
    4 root      RT   0     0    0    0 S  0.0  0.0   0:00.04 migration/1        
    5 root      34  19     0    0    0 S  0.0  0.0   0:00.01 ksoftirqd/1        
    6 root       5 -10     0    0    0 S  0.0  0.0   0:00.03 events/0

Also here the mysql configuration;

[mysqld]
#datadir=/var/lib/mysql
#socket=/var/lib/mysql/mysql.sock
safe-show-database
skip-locking
skip-innodb
query_cache_limit=1M
query_cache_size=200M
query_cache_type=1
max_connections=500
interactive_timeout=100
wait_timeout=100
connect_timeout=10
thread_cache_size=128
key_buffer=16M
join_buffer=1M
max_allowed_packet=16M
table_cache=1024
record_buffer=1M
sort_buffer_size=2M
read_buffer_size=2M
max_connect_errors=10
# Try number of CPU's*2 for thread_concurrency
thread_concurrency=2
myisam_sort_buffer_size=64M
log-bin
server-id=1

#[mysql.server]
#user=mysql
#basedir=/var/lib

[safe_mysqld]
err-log=/var/log/mysqld.log
#pid-file=/var/lib/mysql/mysql.pid
open_files_limit=8192

[mysqldump]
quick
max_allowed_packet=16M

[mysql]
no-auto-rehash
#safe-updates

[isamchk]
key_buffer=64M
sort_buffer=64M
read_buffer=16M
write_buffer=16M

[myisamchk]
key_buffer=64M
sort_buffer=64M
read_buffer=16M
write_buffer=16M

[mysqlhotcopy]
interactive-timeout

And httpd conf;

KeepAlive On
MaxKeepAliveRequests 100
KeepAliveTimeout 15
UseCanonicalName Off
AccessFileName .htaccess
DefaultType text/plain
HostnameLookups Off
ErrorLog logs/error_log
ServerTokens Full


<IfModule prefork.c>
    StartServers 5
    MinSpareServers 5
    MaxSpareServers 10
    MaxClients 150
    MaxRequestsPerChild 0
</IfModule>

[rathin]

Maxclient value is bit low,u may increase to 250

[PeHe]

Hi,

My guess is that Apache does not have enought childs to serve requests.

Change httpd.conf

<IfModule prefork.c>
StartServers 10
MinSpareServers 30
MaxSpareServers 50
ServerLimit 256
MaxClients 200
MaxRequestsPerChild 1000
</IfModule>

Install memcache and APC. Those wont hurt..
After that if I was you I would check IO-wait.

If those wont help you should consider another server for MySQL.
Or perhaps you could setup two node load-balanced cluster..

[bluemaster]

Thanks Pehe and rathin. Now I m changing httpd.conf as Pehe suggested.
Here the new stats before change;

top - 17:29:15 up 1 day, 20:31,  3 users,  load average: 0.30, 0.37, 0.42
Tasks: 264 total,   1 running, 257 sleeping,   0 stopped,   6 zombie
Cpu(s): 44.3% us,  8.7% sy,  0.0% ni, 46.8% id,  0.0% wa,  0.2% hi,  0.2% si
Mem:   4031480k total,  3077484k used,   953996k free,   380696k buffers
Swap:  4192924k total,        0k used,  4192924k free,  1176504k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND            
15010 mysql     15   0  263m  58m 4120 S 71.8  1.5  48:37.91 mysqld             
25906 nobody    16   0 96908 8280 3576 S 10.6  0.2   0:04.02 httpd              
29107 nobody    16   0  138m  51m 3192 S  9.3  1.3   0:00.90 httpd              
28845 nobody    16   0  103m  17m 3524 S  7.6  0.4   0:01.21 httpd              
25347 nobody    16   0 97004 8396 3596 S  0.7  0.2   0:04.98 httpd              
29595 root      16   0  6304 1280  852 R  0.7  0.0   0:00.04 top                
24742 nobody    15   0 96532 8116 3540 S  0.3  0.2   0:02.91 httpd              
24830 nobody    16   0 96932 8276 3624 S  0.3  0.2   0:05.88 httpd              
25517 nobody    16   0 96964 8368 3608 S  0.3  0.2   0:04.42 httpd              
27085 nobody    15   0 96956 8312 3560 S  0.3  0.2   0:02.29 httpd              
27095 nobody    15   0 96760 8144 3588 S  0.3  0.2   0:04.47 httpd              
28816 nobody    16   0 96400 7640 3448 S  0.3  0.2   0:01.58 httpd              
28874 nobody    16   0 96932 8264 3540 S  0.3  0.2   0:00.79 httpd              
29196 nobody    15   0 96372 7708 3416 S  0.3  0.2   0:00.34 httpd              
29206 nobody    15   0 48724 6996 3100 S  0.3  0.2   0:00.22 httpd              
29210 nobody    16   0 96348 7600 3484 S  0.3  0.2   0:00.45 httpd              
29259 nobody    15   0 96396 7776 3412 S  0.3  0.2   0:00.78 httpd

Originally Posted by PeHe

Hi,

My guess is that Apache does not have enought childs to serve requests.

Change httpd.conf

<IfModule prefork.c>
StartServers 10
MinSpareServers 30
MaxSpareServers 50
ServerLimit 256
MaxClients 200
MaxRequestsPerChild 1000
</IfModule>

Install memcache and APC. Those wont hurt..
After that if I was you I would check IO-wait.

If those wont help you should consider another server for MySQL.
Or perhaps you could setup two node load-balanced cluster..

[bluemaster]

Now server started to be slow.. Latest top results:

Tasks: 311 total,   2 running, 303 sleeping,   0 stopped,   6 zombie
Cpu(s): 18.3% us,  7.4% sy,  0.0% ni, 73.6% id,  0.2% wa,  0.2% hi,  0.3% si
Mem:   4031480k total,  3227236k used,   804244k free,   381564k buffers
Swap:  4192924k total,        0k used,  4192924k free,  1254108k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND            
15010 mysql     15   0  263m  68m 4184 S 41.5  1.7  78:11.63 mysqld             
 6334 nobody    15   0 96680 8212 3500 S  1.7  0.2   0:01.65 httpd              
 5579 nobody    15   0 97136 8452 3528 S  0.7  0.2   0:02.53 httpd              
 5529 nobody    16   0 96416 7640 3432 S  0.3  0.2   0:01.04 httpd              
 5577 nobody    15   0 96368 7592 3440 S  0.3  0.2   0:00.58 httpd              
 5587 nobody    16   0 96852 8136 3528 S  0.3  0.2   0:02.39 httpd              
 5682 nobody    16   0 96472 7740 3472 S  0.3  0.2   0:01.09 httpd              
 6050 nobody    15   0 96652 7864 3448 S  0.3  0.2   0:00.53 httpd              
 6159 nobody    16   0 96580 7928 3556 S  0.3  0.2   0:00.78 httpd              
 6164 nobody    15   0 96368 7776 3428 S  0.3  0.2   0:00.93 httpd              
 6176 nobody    16   0 96352 7468 3412 S  0.3  0.2   0:00.65 httpd              
 6178 nobody    16   0 96404 7700 3500 S  0.3  0.2   0:01.31 httpd              
 6212 nobody    15   0 52640  10m 3000 S  0.3  0.3   0:00.58 httpd              
 6316 nobody    16   0 95996 6928 3152 S  0.3  0.2   0:00.30 httpd              
 6332 nobody    16   0 96356 7532 3380 S  0.3  0.2   0:00.51 httpd              
 6393 nobody    16   0 96520 7756 3404 R  0.3  0.2   0:00.56 httpd              
 6484 nobody    15   0 96388 7752 3404 S  0.3  0.2   0:00.70 httpd

[bluemaster]

load average: 1.57, 0.81, 0.48

[The Universes]

I had that kind of problem with a client who also had large number of concurrent connections. I eventually gave up on Apache and moved him to Lighttpd and that fixed the problem (load avg: 10 -> 3)

The problem is that Apache spawns a lot of child processes to handle those connections, which seems to drive the load up no matter what.

[Klentelaris]

I have also problem with high load and memory usage.Load goes up to 10,and sites are static.When i before had cron running which rotate images,load went up to 56 and i barely loaded whm then.And memory usage is 80%.Howmuch i know load should'nt not be bigger then 2 on dual core proccesor(or 4 logicly on 4 core proccesor)

[RalphC]

Try this script.
http://rfxnetworks.com/sim.php

Also, check what is using up your MYSQL on your forums. Do you have anything server intensive like a chatroom or a new modification?

[Klentelaris]

Originally Posted by RalphC

Try this script.
http://rfxnetworks.com/sim.php

Also, check what is using up your MYSQL on your forums. Do you have anything server intensive like a chatroom or a new modification?

I installed that,but some thing dont understood there.What means "renice if warning or critical"?

[Klentelaris]

Originally Posted by The Universes

I had that kind of problem with a client who also had large number of concurrent connections. I eventually gave up on Apache and moved him to Lighttpd and that fixed the problem (load avg: 10 -> 3)

The problem is that Apache spawns a lot of child processes to handle those connections, which seems to drive the load up no matter what.

Btw is it possible to install first apache and then later upgrade to lighthttpd?

[Klentelaris]

I managed to resolve load issue temporaly-even i removed cron which require mysql database,it seems script still connect to mysql database even when cron is not active.So i entered wrong info into mysql connect and now load finaly normalized.Luckily,site works even when mysql is not connected.Now i need to install fresh script and put fresh data into(since db size was 400 gb no wonder why it was such load)

[River Marz Media]

A 400GB MySQL Database? Wow, biggest i have seen yet. No wonder things are slow for you. What exactly is that database?