Results 1 to 19 of 19
  1. #1
    Join Date
    Oct 2003
    Location
    Hanoi
    Posts
    4,306

    server for image hosting

    I have a customer asking for a server using for images hosting. His site is quite busy. top command shows around 100 tasks running most of the time.

    I wonder if there is any recommendation about the server hardware? Pls remember he's using dual 265 with raid1 sata disks already. And it seems not enough.

    Also, how about the apache config? Which should be the best optimization for images hosting?

    thanks,

  2. #2
    Join Date
    Mar 2003
    Location
    California USA
    Posts
    13,294
    Why not move away from apache to something like lighttpd?
    Steven Ciaburri | Industry's Best Server Management - Rack911.com
    Software Auditing - 400+ Vulnerabilities Found - Quote @ https://www.RACK911Labs.com
    Fully Managed Dedicated Servers (Las Vegas, New York City, & Amsterdam) (AS62710)
    FreeBSD & Linux Server Management, Security Auditing, Server Optimization, PCI Compliance

  3. #3
    Join Date
    Mar 2006
    Location
    Reston, VA
    Posts
    3,132
    whats the load on the box? what are the apache client settings? i.e MaxClients, minstartclients, maxstart ect. Php? or static content?

    a properly tuned server can server hundreds of pages. You might want to find a engineer that can tune/optimize the server. We have clients pushing around 800 requests eing processed with ~200 requests/second with a load around .3 on a box. just a raid 1 scsi, dual xeon 3.0 with 2gb ram.. and optimized to a T. Box is also doing peek during the day around 90Mbps. And can go much higher if he wanted to as well.

  4. #4
    Join Date
    Oct 2003
    Location
    Hanoi
    Posts
    4,306
    Have tried 2 differences

    Timeout 300
    KeepAlive On
    MaxKeepAliveRequests 10
    KeepAliveTimeout 10
    MinSpareServers 5
    MaxSpareServers 20
    StartServers 8
    MaxClients 256
    MaxRequestsPerChild 10


    Timeout 15
    KeepAlive Off
    MaxKeepAliveRequests 10
    KeepAliveTimeout 10
    MinSpareServers 5
    MaxSpareServers 20
    StartServers 8
    MaxClients 256
    MaxRequestsPerChild 10000

    the load is from 2 to 4x on the peak.

    For the lighttpd, he wants to use server with DirectAdmin, and I am not sure if DA can manage it. Checking with John now

  5. #5
    Join Date
    Aug 2000
    Location
    Sheffield, South Yorks
    Posts
    3,480
    I would assume that it's not just plain images they are serving and that it is a PHP/mySQL based solution running on the box. As a Dual 265 should have no issue with just static images, to be honest, it shouldn't have that much issue with non-static content either. Are you using mySQL query caching? Are images being stored in the database or the filesystem? Does the site make use of caching where it can? Is mySQL tuned?

    With those Apache configs, my personal choice would be to turn keepalive on, but with a 5s or so timeout on it. I'd also handle more requests per child as long as you're not seeing issues with memory leaks etc.
    Karl Austin :: KDA Web Services Ltd.
    UK Business Hosting and Managed Servers - Hosting for Business Users :: 0800 5429 764
    Call us today and ask about our hosting solutions.

  6. #6
    Join Date
    Feb 2004
    Location
    Atlanta, GA
    Posts
    5,627
    Quote Originally Posted by gate2vn
    Have tried 2 differences

    Timeout 300
    KeepAlive On
    MaxKeepAliveRequests 10
    KeepAliveTimeout 10
    MinSpareServers 5
    MaxSpareServers 20
    StartServers 8
    MaxClients 256
    MaxRequestsPerChild 10


    Timeout 15
    KeepAlive Off
    MaxKeepAliveRequests 10
    KeepAliveTimeout 10
    MinSpareServers 5
    MaxSpareServers 20
    StartServers 8
    MaxClients 256
    MaxRequestsPerChild 10000

    the load is from 2 to 4x on the peak.

    For the lighttpd, he wants to use server with DirectAdmin, and I am not sure if DA can manage it. Checking with John now

    out of curiosity why do you need a control panel on an image only server?

  7. #7
    Join Date
    Oct 2003
    Location
    Hanoi
    Posts
    4,306
    I am not the admin of this server Just help him to check his current server before switching. And he has some small websites that need control panel.

    Karl: yes, it's not only HTML and images. It's PHP/mySQL. But all images are stored in file system, not in database. There are paths to images location in database only. The mysql has been tuned already. also, the sysctl.conf has been optimized as well

  8. #8
    Join Date
    Aug 2000
    Location
    Sheffield, South Yorks
    Posts
    3,480
    In which case, I'd say they would be better off getting a 2nd server and running mySQL from one and the web frontend from the other. Probably keeping the mySQL on the current box and moving the front-end to a Dual Core of some description. Do you know how much RAM is in the current server and if they are using query caching? QC can really drop the loads on a box, we enabled only a small 32MB cache on an ecommerce site and the loads dropped by half.
    Karl Austin :: KDA Web Services Ltd.
    UK Business Hosting and Managed Servers - Hosting for Business Users :: 0800 5429 764
    Call us today and ask about our hosting solutions.

  9. #9
    Join Date
    Oct 2003
    Location
    Hanoi
    Posts
    4,306
    yes, I have put this into his my.cnf

    query_cache_type=1
    query_cache_limit=1M
    query_cache_size=32M

    The server has 2GB RAM. Currently, the load is around 2, but not many processes. I just changed the timeout value to 5, as your suggestion, and see how it's working

    thanks,

  10. #10
    Join Date
    Aug 2000
    Location
    Sheffield, South Yorks
    Posts
    3,480
    Next step, I'd install something like mytop (top for mySQL) and have a look in that (easier to get a realtime overview rather than running the commands yourself). You want to be looking for the number of slow queries and trying to indentify which ones they are, you also want to be looking for queries that are executing most often and seeing if they can be optimised (Use: EXPLAIN in front of the query) - if you're getting file sorts then you need to look at sorting the query so that file sort isn't used - as that is a big killer.
    Karl Austin :: KDA Web Services Ltd.
    UK Business Hosting and Managed Servers - Hosting for Business Users :: 0800 5429 764
    Call us today and ask about our hosting solutions.

  11. #11
    Join Date
    Feb 2004
    Location
    Atlanta, GA
    Posts
    5,627
    Quote Originally Posted by KDAWebServices
    Next step, I'd install something like mytop (top for mySQL) and have a look in that (easier to get a realtime overview rather than running the commands yourself). You want to be looking for the number of slow queries and trying to indentify which ones they are, you also want to be looking for queries that are executing most often and seeing if they can be optimised (Use: EXPLAIN in front of the query) - if you're getting file sorts then you need to look at sorting the query so that file sort isn't used - as that is a big killer.

    Rather than trying to catch them with mytop why not log them?
    you can add the following into your /etc/my.cnf
    log-slow-queries = /var/lib/mysql/slow.log
    log-queries-not-using-indexes

  12. #12
    Join Date
    Aug 2000
    Location
    Sheffield, South Yorks
    Posts
    3,480
    I do it with mytop, as I like to see what's going on with qps and qcache hits etc. when those queries are running. Never had a need to log them, but yes, it's an option.
    Karl Austin :: KDA Web Services Ltd.
    UK Business Hosting and Managed Servers - Hosting for Business Users :: 0800 5429 764
    Call us today and ask about our hosting solutions.

  13. #13
    Join Date
    Nov 2001
    Location
    Atlanta, GA
    Posts
    632
    Karl's right. The definition of a slow query can vary, so you want to see how fast the server's pumping them out and how long they're taking to complete. Some tips are:
    - Use InnoDB whenever possible (which should be any place you're not using FULLTEXT indexing)
    - Get rid of all filesorts
    - Remove any queries you don't need and cache to static files instead

    I'd bump that query cache up to 64M or even higher. Also, if possible, upgrade to Apache 2.2.x and make sure you have sendfile and mmap support enabled. KeepAlive should *always* be off.

    Some other questions:
    What's the contents of the rest of your my.cnf?
    What is the output from a SHOW STATUS; query?
    Former owner of A Small Orange
    New owner of <COMING SOON>

  14. #14
    Join Date
    Oct 2003
    Location
    Hanoi
    Posts
    4,306
    Havent installed mytop yet. I usually use slow logs. Might give it a try now

    Karl: I am not sure, but after changing timeout to 5, it seems the apache stuck after few hours. In error_log, there is nothing saying why. It's simply not responding!

    also, I guess the query cache going to 64M seems help a bit. I have checked qcache, and seen Qcache_lowmem_prunes is very high. Just edit the query cache value and keep an eye on it. Thanks for reminding about the qcache

    Timdorr: I am not sure when you write "cache to static files instead". How can it be done?

    thanks,

  15. #15
    Join Date
    Oct 2003
    Location
    Hanoi
    Posts
    4,306
    another thing is CPU load in apache status

    CPU Usage: u2459.81 s355.78 cu.05 cs.11 - 95.3% CPU load
    14.2 requests/sec - 88.4 kB/second - 6.2 kB/request
    14 requests currently being processed, 11 idle servers

    See, there are not many requests now, but the CPU load is going up to 95.3%!!! Meanwhile the server load is still fine
    # uptime
    07:41:51 up 47 days, 9:15, 2 users, load average: 1.98, 1.71, 1.73

    is there any idea why the CPU load is so high?

    thanks,

  16. #16
    Join Date
    Aug 2000
    Location
    Sheffield, South Yorks
    Posts
    3,480
    What's your vmstat output like?
    Karl Austin :: KDA Web Services Ltd.
    UK Business Hosting and Managed Servers - Hosting for Business Users :: 0800 5429 764
    Call us today and ask about our hosting solutions.

  17. #17
    Join Date
    Oct 2003
    Location
    Hanoi
    Posts
    4,306
    # uptime
    17:01:54 up 47 days, 18:35, 1 user, load average: 4.21, 4.63, 4.65

    # vmstat
    procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu----
    r b swpd free buff cache si so bi bo in cs us sy id wa
    2 1 8432 97644 29896 1226256 0 1 57 63 6 6 16 5 77 2

  18. #18
    Join Date
    Aug 2000
    Location
    Sheffield, South Yorks
    Posts
    3,480
    At a quick glance it could be disk i/o that's forming a bottleneck for you.
    Karl Austin :: KDA Web Services Ltd.
    UK Business Hosting and Managed Servers - Hosting for Business Users :: 0800 5429 764
    Call us today and ask about our hosting solutions.

  19. #19
    Join Date
    Oct 2003
    Location
    Hanoi
    Posts
    4,306
    I am thinking about it as well. However, checking mysql status shows me some values can be optimized again. Trying it first But I guess I will need to build a scsi system for him

    thanks for your great helps

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •