Results 1 to 7 of 7
-
11-18-2007, 06:38 AM #1Aspiring Evangelist
- Join Date
- Nov 2007
- Posts
- 359
How to manage massive file storage needs for youtube clone?
I'm interested in knowing how to handle the file storage of a youtube clone?
Is it just a matter of getting more servers with a few hdds or are there specialized companies that one can upload files over a distributed file streaming network?
The reason I ask is because I have thousands of gigabytes of videos and it appears to be impossible to upload it on 1 dedicated server or even a few.
-
11-26-2007, 02:33 AM #2Web Hosting Master
- Join Date
- Jan 2002
- Posts
- 1,400
If you need lot of space, you can use content delivery network, but that will be costly for start up site. Other option is using amazon s3 storage. Video on your own server give more freedom, you can access with ssh and move if needed.
-
11-26-2007, 08:05 PM #3WHT Addict
- Join Date
- Sep 2006
- Posts
- 141
At least a file server (NAS/SAN setup) for the video files.
CDN is an option but costly, the cheapest way to start up is setting up some sort of file server in the same place you got your media/web server.
-
11-27-2007, 03:35 AM #4Newbie
- Join Date
- Jul 2007
- Posts
- 21
I think NAS/SAN is the only way...
-
11-27-2007, 11:02 PM #5Junior Guru Wannabe
- Join Date
- Mar 2004
- Posts
- 52
Yes I would look into the Amazon S3 system. It will allow you to grow the storage infinitely and pay only for what you use in storage + bandwidth.
-
11-27-2007, 11:17 PM #6Web Hosting Master
- Join Date
- Feb 2003
- Location
- Canada
- Posts
- 1,010
NAS/SAN is probably your best solution. There are companies that would specialize in this type of setup but generally you would have a NAS that is fed by 2/3/4 webservers, with a load balancer in front of them proxying requests.
Many other logistics will come in to play, such as the # of files per directory, etc.
-
11-28-2007, 12:03 AM #7Account Suspended
- Join Date
- Oct 2004
- Location
- Nevada
- Posts
- 887
If you want to go cheap, and arent moving much traffic, install a PERC card in your dedicated server and connect it to a Dell (or other) RAID array. You can do 4-5 TB per array without issue for low traffic.
The issue is that storing lots of video files is a money losing proposition.
We have one client with 12 TB of video files, however, they are using multiple servers and multiple RAID arrays. It gets expensive when you start scaling for traffic.