Results 1 to 34 of 34
  1. #1
    Join Date
    Apr 2005
    Posts
    38

    Question Lighttpd or thttpd? for many images of 100k size

    Hi everybody,

    I plan to start website with big online gallery. These images will be jpegs of size about 100k each, it will be just gallery - no sharing or uploading.

    What should I use as web-server?

    Website
    My idea is to server dynamic pages with apache, I am used to it, it works

    Images
    However, from what I have read everywhere there are much better ways to server images.
    So what should I use? Lighttpd or thttpd or something else?
    Images will be on separete ip.

    Thumbails
    I also plan to make it possible for visitors to use small thumbails (size about 3 - 7kb) as link image on their websites. There could lot of hits for this. This can be also on different ip, and I can use different webserver than large images - if it is better.

    THanks a lot for any suggestions.


    Regards,
    Elliot.

  2. #2
    Join Date
    Jul 2006
    Posts
    1,078
    Why not just stick with Apache

  3. #3
    Join Date
    Dec 2004
    Location
    New York, NY
    Posts
    10,574
    Quote Originally Posted by Linuxtechie
    Why not just stick with Apache
    Apache is highly inefficient for serving static files.

  4. #4
    Join Date
    Aug 2006
    Location
    Ireland
    Posts
    813
    wikipedia uses lighttpd to serve its images
    youtube use lighttpd to stream its videos

    hell we use lighttpd to serve files

    lighttpd its faster than apache for dynamic content as well but starting of serving static files is a good idea

    i dont no bout thttpd tho never tried it

  5. #5
    Join Date
    Jun 2004
    Location
    Bay Area
    Posts
    1,320
    Lighttpd is much more advanced than thttpd. It has optional plugins that you can use for, ie, anti-hotlinking. With the new v1.5 of Lighttpd async IO will also be supported, which will boost performance when serving a lot of static files.

  6. #6
    Join Date
    Dec 2002
    Location
    chica go go
    Posts
    11,858
    Serving up to 16gb of image per day, my server load was up around 0.60 when i was running thttpd. Now it's down at 0.00 running litthttpd.

  7. #7
    Join Date
    Dec 2006
    Posts
    34
    Does lighthttpd work with cPanel? If it does, does anyone have a very detailed tutorial as to how someone would go about replacing Apache 1.3.37 with lighthttpd?

    If it doesn't, oh well.

  8. #8
    Join Date
    Dec 2006
    Posts
    477
    As far as I know, lighttpd does not work with any of the major control panels at the moment. However the development trunk of the open source ISPConfig panel does support it and it will be included in their next release.

    I'm currently moving a php application over to it as it is much more memory efficient than apache. lighttpd sits there in its own single thread using 4Mb then farms out the php requests to a pool of 20 (configurable) php server threads each using 20Mb. (12Mb for PHP plus an 8Mb Opcode cache from the xcache accelerator). This means my server needs 404Mb of memory for the web server be there 1 connection or 100 simulatenous connections (lighttpd will queue requests if there aren't backend php servers available unlike apache which will spawn more threads).

    Under apache each thread required 32Mb of ram. Therefore 100 simultaneous connections required over 3Gb of ram, which caused paging. Lighttpd has the benefit that the 20 php threads running in their own process are only used when a php page is being processed and static files are served directly out of the lighttpd thread. Whereas under apache, once a thread had the php code loaded into memory, that thread consumed that much memory until it was recycled, even if it was only serving static images.

    A nice side effect of having the fixed pool of 20 php threads is I now know mysql will never get more than 20 simultanous connections so I can safely allocate more memory to per-connection buffers in the mysql configuration, whereas before I had to cater for as connections as I had set the apache thread limit to, and the server is far less susceptable to thrashing from too many queries running at once.

    One downside to consider - the core of lighttpd is single threaded, so if all you are doing is serving static pages and you have multiple processors/cores, then it isn't going to make the best use of your resources (although it will still probably outperform apache, because apache is that bad). The commerical lightspeed server will be faster for highend machines. If you are yet to buy a box, one fast single core processor is better for lighttpd and static images. In a situation like mine where you have php pages, the php is running is its own pool of 20 processes so you do get full benefits of multiple processors.

    Finally if you can live with its feature limitations, tux on redhat/centos is much faster than anything else available as it runs in kernel space, at least according to the benchmarks by the author of lighttpd.

  9. #9
    Join Date
    Aug 2006
    Location
    Ireland
    Posts
    813
    As far as I know, lighttpd does not work with any of the major control panels at the moment.
    you got that wrong! lets rephrase that! control panels dont work with lighttpd lol

    control panels like plesk are holding back the industry by using outdated components, for example i used plesk over a year ago and it had mysql3 by default! it was impossible to upgrade to php5 as well, ( it could have changed by now) but overall control panels restrict your choices

    rbbot is correct apache does eat ram like an elephant

  10. #10
    Join Date
    Jan 2007
    Location
    Russia
    Posts
    79
    rbbot is correct apache does eat ram like an elephant
    yeah

    And have you tried nginx?

  11. #11
    Join Date
    Dec 2004
    Location
    New York, NY
    Posts
    10,574
    for example i used plesk over a year ago and it had mysql3 by default!
    Plesk doesn't control what version of Apache, MySQL, PHP, etc. that you are running, it will simply use whatever comes with the system (from the fresh RedHat/CentOs install).

    you got that wrong! lets rephrase that! control panels dont work with lighttpd lol
    I have read things about people using ISPConfig (a free control panel) with their Lighttpd setups. I'm actually looking to try this out myself.

  12. #12
    Join Date
    Aug 2006
    Location
    Ireland
    Posts
    813
    Plesk doesn't control what version of Apache, MySQL, PHP, etc. that you are running, it will simply use whatever comes with the system (from the fresh RedHat/CentOs install).
    that is true, but as i said sait it happened over a year ago and i dont know (or care cli does me fine) if it changed now, trying to upgrade mysql/php in that case broke plesk

  13. #13
    Join Date
    Dec 2006
    Posts
    477
    Plesk 8.1 is compatible with mysql 5, but if mysql 5 isn't already on your system when you install it, the plesk installer still downloads and installs version 4.1 (and this was when yum update would install 5.0.22). Their release notes don't say anything positive or negative about php 5.

    I recently did a test using MS Virtual PC, did a centos 4.4 install, enabling the yum source for centos plus packages and using yum update after installing plesk to install mysql 5.0.22 and php 5.1.6 and it seems to run ok. The webmail didn't work but I used the Virtual PC undo disks to roll it back to before upgrading the mysql/php components and it didn't work then either, so its probably not the upgrade the caused the problem.

    The 8.1 release seems better than the old 7.5 releases - at least it is now putting applications like php and mysql in their normal places on the system so they can be updated easily rather than burying them in its own directory structure. However, I've yet to try it in production, only on a test VM.

  14. #14
    I'd like to throw in with this question, I've personally tried to use lighttpd with apache, where lighttpd was serving static content (image files) via an Apache ReverseProxy on the same server. This worked for seriously reducing CPU and Memory usage, but it slowed down the response times of the image loading a great deal. For those of you who have successfully setup to use lighttpd to serve static content, how did you set things up?

    Did you run reverse proxy through lighttpd, that then called apache for dynamic content? Or did you tweak lighttpd and apache to serve files more quickly? I have considered testing out an all lighttpd server that also did the php and may do so in the future, but for now I am more interested in using lighttpd to for static content (primarily images).

    Thanks for any insight you might have.

  15. #15
    Quote Originally Posted by InDagda
    I'd like to throw in with this question, I've personally tried to use lighttpd with apache, where lighttpd was serving static content (image files) via an Apache ReverseProxy on the same server. This worked for seriously reducing CPU and Memory usage, but it slowed down the response times of the image loading a great deal. For those of you who have successfully setup to use lighttpd to serve static content, how did you set things up?

    Did you run reverse proxy through lighttpd, that then called apache for dynamic content? Or did you tweak lighttpd and apache to serve files more quickly? I have considered testing out an all lighttpd server that also did the php and may do so in the future, but for now I am more interested in using lighttpd to for static content (primarily images).

    Thanks for any insight you might have.
    I set up lighttpd after Apache1 started choking on serving images (a small image hosting site, no dynamic work, just straight http downloads) even before the bandwidth would start being the issue.

    The loads went from "insane" (>20) to "0.01" during peak hours. That was using the default lighttpd config. lighttpd has been a blessing and a half for me, I was skeptical about PHP/MySQL compatibility with it but I really did not need to do anything to make it work except recompile PHP to enable fastcgi.

    I would definitely recommend it!

    Edit: to give you a better idea of what to expect, this site serves a bit under a million images per day (hits).
    Rsync server backup / RAID6 Arrays - do you know where your data is? Affordable host backup solutions•

    Server monitoring - are you with HyperSpin? Great same service, better prices•

  16. #16
    Join Date
    Oct 2002
    Posts
    702
    I don't know why people think Lighttpd is only useful for images. I've been serving up over 10 million PHP based web pages per day using it without trouble. I've also used thttpd and its been great for getting started quickly. Once you get out of httpd.conf apache hell a whole new world opens up.
    ServerMatingProject.com
    The World's first server mating experiment
    We give new meaning to I/O intensive and hot swap

  17. #17
    Join Date
    Aug 2006
    Location
    Ireland
    Posts
    813
    i wih to back TheVoice up

    it took few minutes to set up php5.2 and php4.4.4 and perl to run lighttpd on same port and ip without any ugly hacks, u just define another cgi handler, its also possible to add ruby as easily but i had no need for it


    php5.2 fastcgi is very fast, i recommend the pecl APC extension as well to spice things up

    lighttpd takes a few seconds to compile while apache takes few minutes

    alot of people have problems initially that they are stuck with the .htaccess mindset but the rewrite rules available are very powerfull, and then theres the fear of the unknown

    overall you have to give the credit to lighttpd developers its one excellent piece of software and v1.5 is around the corner and im eagerly awaiting it due to a pile of features

  18. #18
    Join Date
    Apr 2005
    Posts
    38
    Thanks a lot to all of you! I am much clever now

    So it seems that the most of you recommend to use lighttpd as the only web server. I am sure to use lighttpd to serve images.

    Is there any negative to use ligttpd to server php+mysql pages compared to apache?
    (ok, I will have to learn how to rewrite htaccess, but I mean something more serious)

    The another point to use lighttpd for images and apache for pages was this: when there will be lots of request for images, I can limit lighttpd to keep the server up... so at least normal pages will be still available (served by apache).

    Is there an easy way to have different configuration for images directory (to serve it on subdomain images.domain.com on different ip) and different configuration for dynamic sites with php + mysql (very few queries)?
    Does anybody have experience with this?

    Thanks.

  19. #19
    Join Date
    Mar 2003
    Location
    California USA
    Posts
    13,294
    If you are just hosting images I do not see why you cannot saturate a 100mbit line with images on modest hardware with lighttpd.
    Steven Ciaburri | Industry's Best Server Management - Rack911.com
    Software Auditing - 400+ Vulnerabilities Found - Quote @ https://www.RACK911Labs.com
    Fully Managed Dedicated Servers (Las Vegas, New York City, & Amsterdam) (AS62710)
    FreeBSD & Linux Server Management, Security Auditing, Server Optimization, PCI Compliance

  20. #20
    Join Date
    Apr 2005
    Posts
    535
    whereas i have the opposite problem to everyone else here with lighttpd. with apache my server load never breaks about 0.60 with lighttpd it can easliy go over 1.3 with same number of users

  21. #21
    Join Date
    Aug 2006
    Location
    Ireland
    Posts
    813
    Is there any negative to use ligttpd to server php+mysql pages compared to apache?
    im having a problem one of the new boxes of lighttpd dying every few hours (seems to stop now after a kernel update) without any errors in any of the logs, but thats only one of the dozen servers running lighttpd

    also make sure when compiling to have pcre/pcre-devel libraries installed alot of people miss that even tho its mentioned in the guide as a requirement

    another thing is one of large sites i mantain (no not the one in sig) refuses to run on lighttpd1.4.13 + php5.2, the app causes php5.2 to go wild and eat up all the ram, same application runs fine on lighttpd1.4.13 + php5.1.6 im not sure what the problem seems related to the app itself or php


    The another point to use lighttpd for images and apache for pages was this: when there will be lots of request for images, I can limit lighttpd to keep the server up... so at least normal pages will be still available (served by apache).
    yes you can limit a subdomain in several ways:
    *number if concurent connections (mod_evasive)
    *transfer limit per connection
    *overall transfer limit


    Is there an easy way to have different configuration for images directory (to serve it on subdomain images.domain.com on different ip) and different configuration for dynamic sites with php + mysql (very few queries)?
    yes different subdomains can have different configs


    whereas i have the opposite problem to everyone else here with lighttpd. with apache my server load never breaks about 0.60 with lighttpd it can easliy go over 1.3 with same number of users
    strange, did u tweak your config? is it lighttpd causing all the load or php?

  22. #22
    Join Date
    Mar 2003
    Location
    California USA
    Posts
    13,294
    Quote Originally Posted by NWSTech
    whereas i have the opposite problem to everyone else here with lighttpd. with apache my server load never breaks about 0.60 with lighttpd it can easliy go over 1.3 with same number of users

    I have had that happen aswell. It all depends on what it is doing.
    Steven Ciaburri | Industry's Best Server Management - Rack911.com
    Software Auditing - 400+ Vulnerabilities Found - Quote @ https://www.RACK911Labs.com
    Fully Managed Dedicated Servers (Las Vegas, New York City, & Amsterdam) (AS62710)
    FreeBSD & Linux Server Management, Security Auditing, Server Optimization, PCI Compliance

  23. #23
    Join Date
    Apr 2005
    Posts
    535
    it seems to be all php from what i could see (have gone back to apache at the moment cos of it, but lighttpd is FAR faster so im just curious as to what is causing it)

    P.S. apologies for the thread hijack

  24. #24
    Join Date
    Mar 2003
    Location
    California USA
    Posts
    13,294
    another thing is one of large sites i mantain (no not the one in sig) refuses to run on lighttpd1.4.13 + php5.2, the app causes php5.2 to go wild and eat up all the ram, same application runs fine on lighttpd1.4.13 + php5.1.6 im not sure what the problem seems related to the app itself or php
    Is it the php processes using up the ram? If so its probably just an issue with fastcgi + php memory leaks.
    Steven Ciaburri | Industry's Best Server Management - Rack911.com
    Software Auditing - 400+ Vulnerabilities Found - Quote @ https://www.RACK911Labs.com
    Fully Managed Dedicated Servers (Las Vegas, New York City, & Amsterdam) (AS62710)
    FreeBSD & Linux Server Management, Security Auditing, Server Optimization, PCI Compliance

  25. #25
    Join Date
    Aug 2006
    Location
    Ireland
    Posts
    813
    yea same happened to me as mentioned previously php5.2 fcgi goes wild with this particular site i have

    im not sure whats wrong its 40000lines of php4 mostly code so god knows whats causing it i think one of the extensions is unstable in this case

    If so its probably just an issue with fastcgi + php memory leaks.
    yea i figured as much, not to worry it runs fine with 5.1.6 atm and ill have to sit down and recode it one day eitherways

  26. #26
    Join Date
    Mar 2003
    Location
    California USA
    Posts
    13,294
    Quote Originally Posted by mihd
    yea same happened to me as mentioned previously php5.2 fcgi goes wild with this particular site i have

    im not sure whats wrong its 40000lines of php4 mostly code so god knows whats causing it i think one of the extensions is unstable in this case


    yea i figured as much, not to worry it runs fine with 5.1.6 atm and ill have to sit down and recode it one day eitherways

    Do you compile php with the bare minimum extensions?
    Steven Ciaburri | Industry's Best Server Management - Rack911.com
    Software Auditing - 400+ Vulnerabilities Found - Quote @ https://www.RACK911Labs.com
    Fully Managed Dedicated Servers (Las Vegas, New York City, & Amsterdam) (AS62710)
    FreeBSD & Linux Server Management, Security Auditing, Server Optimization, PCI Compliance

  27. #27
    Join Date
    Aug 2006
    Location
    Ireland
    Posts
    813
    yea i always compile but this site makes use of good few extensions,

    not to worry the problem is not lighttpd related but php

  28. #28
    Join Date
    Apr 2005
    Posts
    535
    its not using all the ram (leaves about 100M free or theirabouts) but the load just skyrockets on the php processes.

  29. #29
    Join Date
    Oct 2002
    Posts
    702
    When the PHP process goes crazy type in gdb /path/to/php pid . That should give you some insight into what extension is causing problems.
    ServerMatingProject.com
    The World's first server mating experiment
    We give new meaning to I/O intensive and hot swap

  30. #30
    Join Date
    Nov 2004
    Posts
    345
    Does anyone know if lighttpd has the Expires header ? That's about my only requirement for serving images. Speeds up everything a lot.

    [edit] nevermind. I see that it does.
    Last edited by digdugdog; 01-24-2007 at 12:56 AM.

  31. #31
    Join Date
    Aug 2006
    Location
    Ireland
    Posts
    813
    here you go a quck php snippet to either show image or send a 304 works in both apache and in lighttpd and doesnt make use of apache_get_headers()

    PHP Code:
        $avatar $User->get('avatar');
        
        
    $image_type $avatar['image_type'];
        
    $image_size $avatar['image_size'];
        
    $image_time $avatar['timestamp'];
        
        
    // Getting headers sent by the client.
        
    if ( 
            isset( 
    $_SERVER['HTTP_IF_MODIFIED_SINCE'] ) 
            && 
            
    strtotime$_SERVER['HTTP_IF_MODIFIED_SINCE'] ) >= $image_time 
        
    ){
                
               
    // Client's cache IS current, so we just respond '304 Not Modified'.
               
    header('Last-Modified: '.gmdate('D, d M Y H:i:s'$image_time).' GMT'true304);
        }
        else {
            
               
    // Image not cached or cache outdated, we respond '200 OK' and output the image.
               
    header('Last-Modified: '.gmdate('D, d M Y H:i:s'$image_time).' GMT'true200);
               
    header("Content-Length: $image_size");
            
    header("Content-Type: $image_type");
            print 
    $avatar['image'];
        } 

  32. #32
    Join Date
    Dec 2006
    Posts
    477
    If you want an example of lighttpd being used for php on a high volume site:

    Response Headers - http://sourceforge.net/index.php
    Transfer-Encoding: chunked
    X-Powered-By: PHP/5.2.0
    Content-Type: text/html
    Date: Wed, 24 Jan 2007 12:29:01 GMT
    Server: lighttpd/1.4.13

    200 OK


    Currently I'm playing with the source to see if I can make it adaptively throttle the bandwidth so that you stay within your bandwidth capacity for the month and don't get charged an overage with your ISP.

  33. #33
    Join Date
    Aug 2006
    Location
    Ireland
    Posts
    813
    Quote Originally Posted by RBBOT

    Currently I'm playing with the source to see if I can make it adaptively throttle the bandwidth so that you stay within your bandwidth capacity for the month and don't get charged an overage with your ISP.
    why? it already has alot of settings to control bandwidth in lighttpd


    see
    http://trac.lighttpd.net/trac/wiki/D...TrafficShaping
    http://trac.lighttpd.net/trac/wiki/s...ectionsDetails

    also this option
    evasive.max-conns-per-ip: upper limit of number of connections per ip allowed.

  34. #34
    Join Date
    Dec 2006
    Posts
    477
    It lets you limit the bandwidth per second. I'm trying to modify that for bandwidth per month. Say you have bought 4mb/s of bandwidth but for the last hour you've only used 2mb/s. That means you could use 6mb/s for the next hour and still stay within your allowance from the provider. But if you limit it to 4mb/s in lighttpd at the moment your are throwing away 900Mb of transfer that you have paid for.

    I plan to have it know exactly how much data it has send since whatever day of the month bandwidth is billed on, and how much data *could* have been sent without breaking the limit, and then only throttle if sending the data this second would push the average for the month over the allowed limit.
    Last edited by RBBOT; 01-24-2007 at 08:57 AM.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •