Results 1 to 9 of 9
  1. #1

    Speed up serving large files?

    Hi,

    I can't seem to find a clear answer for that on the web so I would like some help from you guys!

    I'm working on a web site which will basically be a flash games portal. I have a dedicated server running Apache 2 on a 100mbit dedicated line but my download speed for large files (flash files of over 5mbs) is really slow. I am thinking this is because of Apache but I don't know much about this. I've read that I should change for a lighter http server for serving static files. The way my server is set up is I have 2 virtual machines running, one doing the PHP processing and the other serving static files, both running Apache, so if I have to change HTTP server for the static files it would be very easy. Although I am not sure if this is necessary or if I can tune Apache to push files faster than this.

    Any help will be greatly appreciated.
    Thanks!

    Edit : download speeds are about 150~200kb/s when I know I can get way faster download than this at home.

  2. #2
    Join Date
    Dec 2005
    Posts
    3,110
    With a fair amount of work you can configure Apache to be almost as fast / better than lighttpd at serving static files.

    To be honest you would be better off using lighttpd to start with though.

    Bear in mind Virtualization does bring overheads, and with two VM's youve got two different areas of the disk being accessed constantly, so that will slow things down when you get lots of simultanious connections.

    You might have the throughput there, but you will lack IOPS (I/O operations per second) so you might want to look into a RAID based setup with multiple disks to help alleviate that.

  3. #3
    I was aware that disk IO will probably be my bottleneck here, so I am aiming a lot towards caching in memory (my PHP is made to use the database at a minimum and cache using APC). If it's possible,

    I still think that right now, being the only one accessing my website (it's not public yet), it is weird to only pull 200kb/s out of a server.

    I'd like to be able to cache in RAM all the static files on the second server since it's the only process running on that server, is there any way to do this with lighttpd or apache?

  4. #4
    Join Date
    Mar 2009
    Posts
    2,222
    Quote Originally Posted by Khao8 View Post
    I'd like to be able to cache in RAM all the static files on the second server since it's the only process running on that server, is there any way to do this with lighttpd or apache?
    Yes, see Apache documentation one memory caching; for example:

    Code:
    # Enable memory caching
    CacheEnable mem /
    
    # Limit the size of the cache to 1 Megabyte
    MCacheSize 1024
    But as it's the only process in the second virtual machine, is there any reason why you don't just run two Apache instances on the real hardware?

    Apache should be able to drive a 100mbps network link pretty much flat out.
    Last edited by tim2718281; 11-09-2009 at 09:43 PM.

  5. #5
    I went with virtualization to make backup easier (with LVM partitions) after completely destroying my machine and had to reinstall. I also wanted to split the static content from the dynamic content to make it easier to manage but if I can run 2 instances of apache on the same machine I might reconsider going this way..

  6. #6
    Join Date
    Mar 2009
    Posts
    2,222
    Quote Originally Posted by Khao8 View Post
    I went with virtualization to make backup easier (with LVM partitions) after completely destroying my machine and had to reinstall. I also wanted to split the static content from the dynamic content to make it easier to manage but if I can run 2 instances of apache on the same machine I might reconsider going this way..
    By the way, have you checked the Apache logs to see if they show anything unusual when the slow transfers are in progress?

    They probably won't, but you never know.

  7. #7
    Join Date
    Nov 2003
    Location
    Marylebone, London, UK
    Posts
    530
    If you're simply serving a few file types, may be easier to
    use Nginx instead. It's more easily configured to silently
    drop any requests which don't match one or more specified
    file types. That also saves time on 404s (if you haven't
    already dropped them).

  8. #8
    Join Date
    Apr 2009
    Location
    Nevada
    Posts
    662
    Use mod_gzip in apache. Its implementation allows for using the compression method gzip for a significant reduction of the volume of web page content served over the http protocol.
    James B
    WWW.EZEELOGIN.COM |Setup your Secure Linux SSH GatewayMEET PCI DSS & ISO 27001 Compliance|Manage & Administer Multiple Linux Servers Quickly & Securely.

  9. #9
    Quote Originally Posted by Khao8 View Post
    Hi,

    I can't seem to find a clear answer for that on the web so I would like some help from you guys!

    I'm working on a web site which will basically be a flash games portal. I have a dedicated server running Apache 2 on a 100mbit dedicated line but my download speed for large files (flash files of over 5mbs) is really slow. I am thinking this is because of Apache...
    First of all you should try to use SSHFS (better) or FTP to download your files to test the hardware.

Similar Threads

  1. Serving Files with PHP
    By Adam Hallett in forum Programming Discussion
    Replies: 9
    Last Post: 12-29-2007, 12:44 AM
  2. Robust Cluster for Serving Files
    By -t7- in forum Dedicated Server
    Replies: 9
    Last Post: 03-09-2007, 12:07 PM
  3. Serving large files
    By Force in forum Hosting Security and Technology
    Replies: 10
    Last Post: 05-04-2005, 09:17 PM
  4. serving large files through apache
    By eL-ankebut in forum Dedicated Server
    Replies: 11
    Last Post: 05-01-2005, 10:46 AM
  5. Serving WMV Files... grrrr
    By Michael_Tulsa in forum Dedicated Server
    Replies: 4
    Last Post: 10-21-2004, 08:44 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •