Results 1 to 7 of 7
  1. #1
    Join Date
    Oct 2004
    Location
    Niagara Falls, Ontario
    Posts
    57

    Mean Download Server Specs Please

    Hello all, I'm starting a file download site. I've done a lot of research and I'm currently planning a head for growth and scaling. Looking to serve around 250-500 thousand 5MB files a day.

    I would like some input from people that KNOW what they are talking about, hopefully people that have hosted/ran similar sites.

    The main question I need answered is what will be the first bottle neck for a single download server will run into when delivering the following.

    File size = 10MB.
    Number of downloads = As many 1.2mbit download streams as possible.

    Example Server, lets say...
    Intel 5405
    8GB Ram
    2x640GB RE3 Hardware Raid 1
    1000mbit

    Am I correct in assuming the bottle neck will be the HD's here? So would I be right in assuming this could handle around 200 concurrent downloads @ 1.2mbit(250mbit)?

    Thanks for your help!
    David Sontag

  2. #2
    Join Date
    Sep 2009
    Location
    /home
    Posts
    248
    Yes that will most likely be the case.

    Going with 4x250gb hdds in RAID10 would give you almost the same amount of disk space as your specifications (500gb, so 140gb less), and you will have much better I/O speeds.

    You would probably be able to squeeze in that many downloads with your chosen specifications, but it would be stressful on the HDD's and possibly slow it down too much. Choosing a RAID10 config will eliminate this until you get to larger file sizes and more traffic. In that case you can move to SAS 15K rpm hdds in RAID10.

  3. #3
    Join Date
    Dec 2004
    Posts
    209
    I currently ordered a
    Supermicro 5015B-mTB
    Intel Xeon Quadcore X3220 2,40 ghz 8mb 1066mhz
    4 gigabyte ram
    4 x 2000 gigabyte SATA II HDD
    Hardware RAID-10
    250 mbit unmetered
    I choose Raid10 so it can handle an link upgrade later.

    If your site get popular you might need more space than 640Gb i Raid1
    Better to have too much than too little?

    Also if you are going to run it on a Gbit link you might need to get something else than Apache, etc nginx or lightspeed

    I am trying out lightspeed now because of the web interface, and easy cgi support.

    There also the right filesystem, Kernel with AIO support, web server with AIO etc. Alot of factors Watch out for IO Waits.
    Busy, busy, busy

  4. #4
    Join Date
    Oct 2004
    Location
    Niagara Falls, Ontario
    Posts
    57
    Quote Originally Posted by chrda View Post
    I currently ordered a
    Supermicro 5015B-mTB
    Intel Xeon Quadcore X3220 2,40 ghz 8mb 1066mhz
    4 gigabyte ram
    4 x 2000 gigabyte SATA II HDD
    Hardware RAID-10
    250 mbit unmetered
    I choose Raid10 so it can handle an link upgrade later.

    If your site get popular you might need more space than 640Gb i Raid1
    Better to have too much than too little?

    Also if you are going to run it on a Gbit link you might need to get something else than Apache, etc nginx or lightspeed

    I am trying out lightspeed now because of the web interface, and easy cgi support.

    There also the right filesystem, Kernel with AIO support, web server with AIO etc. Alot of factors Watch out for IO Waits.


    Thanks for the reply, the files will mostly be small and I was thinking of having multiple download servers once I hit around the 250mbit mark on each one. Would be a lot easier to handle a download server going down that way.
    David Sontag

  5. #5
    Join Date
    Dec 2004
    Posts
    209
    I am going for the same, when they 250 a new is ordered
    Considering 12x2Tb servers also.. Raid-10 or Raid-5
    Busy, busy, busy

  6. #6
    Join Date
    Jul 2009
    Posts
    126
    Maybe this is pretty obvious but if you are going for the whole world as market don't foreget to spread your servers around the world too.

    Another tip could be to use some kind of memcache module, like mod_mem_cache for apache or mod_memcache for lighttpd. This could help you with the I/O performance. If you could squeeze in, let's say, the 1000 most dowloaded files (1000*5MB=5GB) in memory this would reduce the stress alot on the HDDs.

  7. #7
    Join Date
    Jun 2006
    Location
    Europe
    Posts
    632
    if you are going to setup for example nginx, you dont need that much RAM, you will be fine with 2-4gb RAM.

    then if you save some money on RAM, you could go with higher raid levels, so you have better io performance, and that is most important for file hosting.

    mental estimate of RAM usage for 200x1.2mbit on nginx, would be something like 500mb of memory usage. maybe even less.
    Last edited by xtrac568; 10-20-2009 at 12:20 PM.

Similar Threads

  1. Need a server built? 2U UK Server Building, All Specs Xeons/Atoms/Core2/AMD etc...
    By keywolf in forum Other Web Hosting Related Offers
    Replies: 3
    Last Post: 05-09-2009, 06:14 AM
  2. Replies: 2
    Last Post: 04-23-2009, 02:58 PM
  3. Specs for a download server
    By Bakie in forum Dedicated Server
    Replies: 27
    Last Post: 06-06-2005, 08:57 AM
  4. Specs for a download server
    By RangerOfFire in forum Dedicated Server
    Replies: 3
    Last Post: 03-14-2005, 11:07 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •