Results 1 to 21 of 21
  1. #1

    Need Advice on High load of the server

    Hi,

    I have a dedicated server with the following configuration

    Intel P4 3ghz
    2GB RAM

    200GB HDD 7200 RPM

    I have two major websites on the server, 25k uniqs for both the websites.The websites use PHP and mysql. During the peak hours the load is going crazy between 30-40 on uptime...

    [[email protected] conf]# uptime
    14:13:48 up 6 days, 6:40, 2 users, load average: 20.43, 30.00, 36.16

    I see that httpd processes take lot of cpu load. Do you guys think its the time to upgrade to higher configuration..

    -H

  2. #2
    It's likely the scripts are highly inefficient and perform recursive SQL lookups.

    It may be a good idea to employ an additional server for MySQL or upgrade, however I'd highly advise either optimising those scripts, or paying someone to do so and tweaking /etc/my.cnf

  3. #3
    jeff

    thanks for the quick response...here is my /etc/my.cnf

    [mysqld]
    safe-show-database
    innodb_data_file_path=ibdata1:10M:autoextend
    datadir=/var/lib/mysql
    socket=/var/lib/mysql/mysql.sock
    max_connections=250
    # MySQL 4.x has query caching available.
    # Enable it for vast improvement and it may be all you need to tweak.
    query_cache_type=1
    query_cache_limit=1M
    query_cache_size=64M
    # log slow queries is a must. Many queries that take more than 2 seconds.
    # If so, then your tables need enhancement.
    log_slow_queries=/var/log/mysqld.slow.log
    long_query_time=2
    # Disable Remote connections
    skip-networking
    [mysql.server]
    user=mysql
    basedir=/var/lib
    [safe_mysqld]
    err-log=/var/log/mysqld.log
    pid-file=/var/run/mysqld/mysqld.pid


    please advice

  4. #4
    Join Date
    May 2006
    Posts
    111
    What about some data on the apache connections like how much bandwidth it is giving out and how many clients connected?

  5. #5
    at peak times I saw ps aux around 270 httpd processes.

    The monthly bandiwidth is like

    [[email protected] conf]# vnstat -m
    eth0
    month rx | tx | total
    ------------------------+---------------+---------------
    Aug '06 106,274 MB | 1,373,415 MB | 1,479,689 MB
    Sep '06 87,192 MB | 1,087,471 MB | 1,174,663 MB
    ------------------------+---------------+---------------
    estimated 126,824 MB | 1,581,776 MB | 1,708,600 MB



    here is the top output...this is just top output during non-peak hours

    top - 15:21:19 up 6 days, 7:47, 2 users, load average: 3.66, 3.10, 3.16
    Tasks: 230 total, 2 running, 227 sleeping, 0 stopped, 1 zombie
    Cpu(s): 10.5% us, 3.3% sy, 0.0% ni, 65.8% id, 19.1% wa, 0.4% hi, 0.9% si
    Mem: 2009092k total, 1755672k used, 253420k free, 73996k buffers
    Swap: 2031608k total, 144604k used, 1887004k free, 1025540k cached
    PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
    8165 apache 16 0 39144 19m 22m S 5.0 1.0 0:02.01 httpd
    6695 apache 15 0 39212 19m 22m S 3.1 1.0 0:03.50 httpd
    8586 apache 15 0 38168 18m 22m S 2.6 1.0 0:00.26 httpd
    8736 apache 15 0 37756 18m 22m S 2.5 0.9 0:00.59 httpd
    7911 apache 16 0 38588 19m 22m R 2.1 1.0 0:00.24 httpd
    7632 apache 15 0 37736 18m 22m S 1.6 0.9 0:00.53 httpd
    7922 apache 15 0 30312 11m 22m S 1.1 0.6 0:00.65 httpd
    8747 apache 16 0 30456 11m 22m S 0.6 0.6 0:00.06 httpd
    7696 apache 16 0 30556 11m 22m S 0.4 0.6 0:00.92 httpd
    7918 apache 15 0 30204 10m 22m S 0.4 0.6 0:00.07 httpd
    8750 apache 16 0 0 0 0 Z 0.4 0.0 0:00.05 httpd <defunct>
    6770 apache 15 0 38440 19m 22m S 0.3 1.0 0:01.38 httpd
    8898 root 17 0 2632 1060 1664 R 0.3 0.1 0:00.06 top
    6771 apache 16 0 37780 18m 22m S 0.2 0.9 0:00.60 httpd
    7024 apache 16 0 30232 11m 22m S 0.2 0.6 0:00.80 httpd
    7281 apache 16 0 39388 20m 22m S 0.2 1.0 0:02.65 httpd
    7528 apache 15 0 30448 11m 22m S 0.2 0.6 0:00.55 httpd
    8749 apache 16 0 30192 10m 22m S 0.2 0.5 0:00.02 httpd
    19508 mysql 16 0 293m 55m 9832 S 0.1 2.8 0:12.79 mysqld
    12394 root 15 0 0 0 0 S 0.1 0.0 0:01.85 pdflush
    5653 apache 15 0 41436 19m 24m S 0.1 1.0 0:01.84 httpd
    6514 apache 15 0 37704 18m 22m S 0.1 0.9 0:01.18 httpd
    6696 apache 16 0 30384 11m 22m S 0.1 0.6 0:00.83 httpd
    7540 apache 15 0 30208 11m 22m S 0.1 0.6 0:02.49 httpd
    7555 apache 15 0 30556 11m 22m S 0.1 0.6 0:00.20 httpd
    7680 apache 15 0 37808 18m 22m S 0.1 0.9 0:01.04 httpd
    7691 apache 15 0 38164 18m 22m S 0.1 1.0 0:00.47 httpd
    7882 apache 15 0 29960 10m 22m S 0.1 0.5 0:00.03 httpd
    7902 apache 16 0 30116 10m 22m S 0.1 0.5 0:00.50 httpd
    8174 apache 16 0 30284 11m 22m S 0.1 0.6 0:00.48 httpd
    8176 apache 16 0 29932 10m 22m S 0.1 0.5 0:00.01 httpd
    8188 apache 15 0 39180 19m 22m S 0.1 1.0 0:00.95 httpd
    8190 apache 15 0 39168 19m 22m S 0.1 1.0 0:02.08 httpd
    8316 apache 15 0 30208 10m 22m S 0.1 0.6 0:00.05 httpd
    8581 apache 15 0 30364 11m 22m S 0.1 0.6 0:00.19 httpd
    1 root 16 0 2724 488 1408 S 0.0 0.0 0:02.41 init
    2 root RT 0 0 0 0 S 0.0 0.0 0:01.43 migration/0

  6. #6
    Join Date
    Dec 2001
    Location
    NYC, NY
    Posts
    798
    well for one you are using ide drives.. there is a reason major sites use scsi.. tweaking your my.cnf wont work.. my guess is you have high i/o load which is bringing the load up
    Blog your life away
    http://photoblog.com

  7. #7
    Join Date
    Dec 2004
    Location
    New York, NY
    Posts
    10,574
    IDE is horrible.

    Go for SCSI 15K
    MediaLayer, LLC - www.medialayer.com Learn how we can make your website load faster, translating to better conversion rates for your business!
    The pioneers of optimized web hosting, featuring LiteSpeed Web Server & SSD Storage - Celebrating 10 Years in Business

  8. #8
    layer0 and zupanm how much of a difference does it make if I use sata harddrive., is sata any better than ide

  9. #9
    Join Date
    Apr 2004
    Location
    San Jose
    Posts
    902
    Try using a PHP accelerator. That will make much more of a difference switching drives, etc.

    eAccelerator, APC, etc. are known to help quite a bit.

    Zend Optimizer is not a PHP accelerator, and is pretty much worthless if you're not using one.
    Specializing in MySQL and website tuning for high traffic sites. cmwsci.com/

  10. #10
    Join Date
    May 2006
    Posts
    111
    I was looking at your top and noticed even at 3.x server load you already have 3/4 of your RAM used. So I would say before even considering anything else look at your RAM/Swap file usage during peak times. If you are heavily swapping then you got your answer there--you need to double RAM or find way to drastically cut your RAM usage...maybe lighttpd or litespeed.

  11. #11
    uname -r
    what kernel version are u using ?!

  12. #12
    hello,

    my OS is Fedora Core release 3 (Heidelberg) and kernel is 2.6.9-1.667smp

  13. #13
    Join Date
    Dec 2001
    Location
    NYC, NY
    Posts
    798
    sata drives will do little since it still uses the cpu for i/o access..

    run a

    vmstat 1 5

    when the load is high so we can see better where the issue might be
    Blog your life away
    http://photoblog.com

  14. #14
    zupanm here is the output of vmstat

    [[email protected] ~]# vmstat 1 5
    procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu----
    r b swpd free buff cache si so bi bo in cs us sy id wa
    50 0 177260 111168 31472 165848 2 3 28 21 23 1 55 7 35 3
    50 0 177260 110848 31476 165844 0 0 24 0 1388 318 95 5 0 0
    50 0 177260 110648 31476 165844 0 0 4 0 1335 226 96 4 0 0
    51 0 177260 106408 31484 166616 0 0 588 0 1412 356 97 3 0 0
    50 0 177260 109240 31488 166612 0 0 212 0 1403 101 99 2 0 0


    also the top output

    top - 08:57:37 up 7 days, 1:24, 2 users, load average: 40.40, 43.35, 40.20
    Tasks: 384 total, 2 running, 382 sleeping, 0 stopped, 0 zombie
    Cpu(s): 79.3% us, 19.5% sy, 0.0% ni, 0.0% id, 0.0% wa, 0.3% hi, 0.8% si
    Mem: 2009092k total, 1952512k used, 56580k free, 33484k buffers
    Swap: 2031608k total, 104804k used, 1926804k free, 221716k cached
    PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
    29784 apache 15 0 35100 22m 22m S 12.1 1.2 0:00.41 /usr/sbin/httpd
    26494 apache 17 0 37960 25m 22m S 8.8 1.3 0:01.28 /usr/sbin/httpd
    31418 apache 15 0 30920 18m 22m S 8.2 0.9 0:00.29 /usr/sbin/httpd
    717 apache 19 0 39272 24m 22m R 6.5 1.2 0:00.20 /usr/sbin/httpd
    31077 apache 15 0 29960 17m 22m S 5.9 0.9 0:01.07 /usr/sbin/httpd
    29319 apache 15 0 37740 25m 22m S 5.6 1.3 0:00.82 /usr/sbin/httpd
    25459 apache 15 0 37860 25m 22m S 5.2 1.3 0:01.23 /usr/sbin/httpd
    26572 apache 15 0 38712 26m 22m S 5.2 1.3 0:02.27 /usr/sbin/httpd
    27808 apache 15 0 30220 18m 22m S 5.2 0.9 0:01.13 /usr/sbin/httpd
    31242 apache 15 0 30028 17m 22m S 5.2 0.9 0:00.26 /usr/sbin/httpd
    27736 apache 15 0 37320 24m 22m S 4.6 1.3 0:01.24 /usr/sbin/httpd
    518 apache 15 0 30204 18m 22m S 4.6 0.9 0:00.23 /usr/sbin/httpd
    24840 apache 16 0 37312 23m 22m S 3.3 1.2 0:00.74 /usr/sbin/httpd
    667 root 17 0 3888 1160 1664 R 2.0 0.1 0:00.24 top -c
    28515 apache 16 0 38736 26m 22m S 1.3 1.3 0:02.81 /usr/sbin/httpd
    676 apache 16 0 29956 17m 22m S 1.0 0.9 0:00.03 /usr/sbin/httpd
    31671 apache 16 0 29716 17m 22m S 0.7 0.9 0:00.26 /usr/sbin/httpd
    25416 apache 15 0 38332 25m 22m S 0.3 1.3 0:05.33 /usr/sbin/httpd
    28210 apache 16 0 37312 24m 22m S 0.3 1.3 0:02.00 /usr/sbin/httpd
    29226 apache 15 0 30280 18m 22m S 0.3 0.9 0:00.50 /usr/sbin/httpd
    30951 apache 15 0 30192 17m 22m S 0.3 0.9 0:00.19 /usr/sbin/httpd
    31439 apache 15 0 34728 22m 22m S 0.3 1.1 0:00.09 /usr/sbin/httpd
    32049 apache 15 0 37688 25m 22m S 0.3 1.3 0:01.69 /usr/sbin/httpd
    32381 apache 16 0 38676 26m 22m S 0.3 1.3 0:01.17 /usr/sbin/httpd
    1 root 16 0 2724 488 1408 S 0.0 0.0 0:02.54 init [3]
    2 root RT 0 0 0 0 S 0.0 0.0 0:01.59 [migration/0]
    3 root 34 19 0 0 0 S 0.0 0.0 0:00.57 [ksoftirqd/0]
    4 root RT 0 0 0 0 S 0.0 0.0 0:01.54 [migration/1]
    5 root 34 19 0 0 0 S 0.0 0.0 0:00.68 [ksoftirqd/1]
    6 root 5 -10 0 0 0 S 0.0 0.0 0:00.49 [events/0]
    7 root 5 -10 0 0 0 S 0.0 0.0 0:00.30 [events/1]
    8 root 6 -10 0 0 0 S 0.0 0.0 0:00.00 [khelper]
    9 root 15 -10 0 0 0 S 0.0 0.0 0:00.00 [kacpid]
    30 root 5 -10 0 0 0 S 0.0 0.0 0:00.66 [kblockd/0]
    31 root 5 -10 0 0 0 S 0.0 0.0 0:00.57 [kblockd/1]
    44 root 13 -10 0 0 0 S 0.0 0.0 0:00.00 [aio/0]
    45 root 5 -10 0 0 0 S 0.0 0.0 0:00.00 [aio/1]

  15. #15
    Join Date
    Dec 2001
    Location
    NYC, NY
    Posts
    798
    can you edit your post and put the top and vmstat outputs in like code /code blocks so the spacing is easier to read
    Blog your life away
    http://photoblog.com

  16. #16
    Sorry for that...'am attaching them as files..Hope that should help you

    vmstat.txt

    topoutput.txt

  17. #17
    Join Date
    Dec 2001
    Location
    NYC, NY
    Posts
    798
    ya the issue is i/o its way too high.. look at the last few lines your at 500 then 200. No amount of tweaking this server will be able to help and sata might just patch the problem till you get bigger.. you will need scsi drives or to continue to patch put the mysql db's on another server with scsi drives
    Blog your life away
    http://photoblog.com

  18. #18
    thanks for the quick response. 'am trying to move to dual core 2.66ghz with sata harddisk. I will give you guys how it goes after the transfer of the server

  19. #19
    Join Date
    Apr 2004
    Location
    San Jose
    Posts
    902
    What makes you think it's doing a lot of IO?

    Both top and vmstat show 0 to 3% iowait. The machine is CPU bound in Apache.

    If you're not using an accelerator, try that.

    If that doesn't help enough, inspect the PHP to see if a better algorithm can be used.

    if that doesn't work, get a faster processor or split over two web machines.
    Specializing in MySQL and website tuning for high traffic sites. cmwsci.com/

  20. #20
    by the way.. this could be a kernel Issue..
    i think updating your kernel.. will be nice for security reasons and for load reasons

  21. #21
    thanks all guys..I will be moving to higher configuration server, I will take all suggestions and try to build the server

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •