hosted by liquidweb


Go Back   Web Hosting Talk : Web Hosting Main Forums : Hosting Security and Technology : How to manage massive file storage needs for youtube clone?
Reply

Forum Jump

How to manage massive file storage needs for youtube clone?

Reply Post New Thread In Hosting Security and Technology Subscription
 
Send news tip View All Posts Thread Tools Search this Thread Display Modes
  #1  
Old
Aspiring Evangelist
 
Join Date: Nov 2007
Posts: 353

How to manage massive file storage needs for youtube clone?


I'm interested in knowing how to handle the file storage of a youtube clone?

Is it just a matter of getting more servers with a few hdds or are there specialized companies that one can upload files over a distributed file streaming network?

The reason I ask is because I have thousands of gigabytes of videos and it appears to be impossible to upload it on 1 dedicated server or even a few.



Sponsored Links
  #2  
Old
Web Hosting Master
 
Join Date: Jan 2002
Posts: 1,363
If you need lot of space, you can use content delivery network, but that will be costly for start up site. Other option is using amazon s3 storage. Video on your own server give more freedom, you can access with ssh and move if needed.

__________________
WebHostingNeeds.com

  #3  
Old
WHT Addict
 
Join Date: Sep 2006
Posts: 141
At least a file server (NAS/SAN setup) for the video files.
CDN is an option but costly, the cheapest way to start up is setting up some sort of file server in the same place you got your media/web server.

Sponsored Links
  #4  
Old
Newbie
 
Join Date: Jul 2007
Posts: 21
I think NAS/SAN is the only way...

  #5  
Old
Junior Guru Wannabe
 
Join Date: Mar 2004
Posts: 52
Yes I would look into the Amazon S3 system. It will allow you to grow the storage infinitely and pay only for what you use in storage + bandwidth.

  #6  
Old
Web Hosting Master
 
Join Date: Feb 2003
Location: Canada
Posts: 945
Quote:
Originally Posted by Gigaron View Post
I'm interested in knowing how to handle the file storage of a youtube clone?

Is it just a matter of getting more servers with a few hdds or are there specialized companies that one can upload files over a distributed file streaming network?

The reason I ask is because I have thousands of gigabytes of videos and it appears to be impossible to upload it on 1 dedicated server or even a few.
NAS/SAN is probably your best solution. There are companies that would specialize in this type of setup but generally you would have a NAS that is fed by 2/3/4 webservers, with a load balancer in front of them proxying requests.

Many other logistics will come in to play, such as the # of files per directory, etc.

  #7  
Old
Account Suspended
 
Join Date: Oct 2004
Location: Nevada
Posts: 887
If you want to go cheap, and arent moving much traffic, install a PERC card in your dedicated server and connect it to a Dell (or other) RAID array. You can do 4-5 TB per array without issue for low traffic.

The issue is that storing lots of video files is a money losing proposition.

We have one client with 12 TB of video files, however, they are using multiple servers and multiple RAID arrays. It gets expensive when you start scaling for traffic.

Reply

Related posts from TheWhir.com
Title Type Date Posted
Dropbox Suffers Temporary Outage Impacting File Sharing and Synchronizing Web Hosting News 2014-03-14 12:11:46
Egnyte Launches Program to PRISM-Proof Enterprise Cloud File-sharing Web Hosting News 2013-09-27 13:45:48
Box Doubles Storage for Free Plan, Launches $5 Enterprise Plan Web Hosting News 2013-08-22 13:32:27
Gladinet Launches Cloud Enterprise Service to Access SAN as a Private Cloud Web Hosting News 2013-03-28 14:19:44
BitTorrent to Launch Cloud Storage Service Web Hosting News 2013-01-25 15:39:23


Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes
Postbit Selector

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off

Forum Jump
Login:
Log in with your username and password
Username:
Password:



Forgot Password?
WHT Host Brief Email:

We respect your privacy. We will never sell, rent, or give away your address to any outside party, ever.

Advertisement:
Web Hosting News:
WHT Membership
WHT Membership



 

X

Welcome to WebHostingTalk.com

Create your username to jump into the discussion!

WebHostingTalk.com is the largest, most influentual web hosting community on the Internet. Join us by filling in the form below.


(4 digit year)

Already a member?