Results 1 to 7 of 7
  1. #1

    How to manage massive file storage needs for youtube clone?

    I'm interested in knowing how to handle the file storage of a youtube clone?

    Is it just a matter of getting more servers with a few hdds or are there specialized companies that one can upload files over a distributed file streaming network?

    The reason I ask is because I have thousands of gigabytes of videos and it appears to be impossible to upload it on 1 dedicated server or even a few.

  2. #2
    If you need lot of space, you can use content delivery network, but that will be costly for start up site. Other option is using amazon s3 storage. Video on your own server give more freedom, you can access with ssh and move if needed.

  3. #3
    At least a file server (NAS/SAN setup) for the video files.
    CDN is an option but costly, the cheapest way to start up is setting up some sort of file server in the same place you got your media/web server.

  4. #4
    I think NAS/SAN is the only way...

  5. #5
    Yes I would look into the Amazon S3 system. It will allow you to grow the storage infinitely and pay only for what you use in storage + bandwidth.

  6. #6
    Join Date
    Feb 2003
    Location
    Canada
    Posts
    1,010
    Quote Originally Posted by Gigaron View Post
    I'm interested in knowing how to handle the file storage of a youtube clone?

    Is it just a matter of getting more servers with a few hdds or are there specialized companies that one can upload files over a distributed file streaming network?

    The reason I ask is because I have thousands of gigabytes of videos and it appears to be impossible to upload it on 1 dedicated server or even a few.
    NAS/SAN is probably your best solution. There are companies that would specialize in this type of setup but generally you would have a NAS that is fed by 2/3/4 webservers, with a load balancer in front of them proxying requests.

    Many other logistics will come in to play, such as the # of files per directory, etc.

  7. #7
    Join Date
    Oct 2004
    Location
    Nevada
    Posts
    887
    If you want to go cheap, and arent moving much traffic, install a PERC card in your dedicated server and connect it to a Dell (or other) RAID array. You can do 4-5 TB per array without issue for low traffic.

    The issue is that storing lots of video files is a money losing proposition.

    We have one client with 12 TB of video files, however, they are using multiple servers and multiple RAID arrays. It gets expensive when you start scaling for traffic.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •