Results 1 to 14 of 14
  1. #1

    What to do when the servr HD is not big enough?

    Hello,

    I am going to launch a site very soon and would like to get ready with any possible problems that could arise.

    I have at the moment 10 000GB bandwidth and 1 server 8 core 2GHZ xenon with 4 GB of ram equipped with 2 HD of 500GB RAID 10 +a back up server of 30GB(at the moment).

    I really need some expert advise on what I could organize or do if things pick up very fast.

    I will be hosting mp3s and videos, my marketing will be pretty this is why I wish to prepare myself now.

    I explain how the things work right now:

    My website and database are on this server, at the start I will only have a shared 100Mbps pipe going through it with limitation of 10000GB for the bandwidth, my backups are done on a daily basis on the raid 10 HDs but only weekly for the remote(to limit the bandwidth usage).
    What would you do if my 2 500gb HDs are not enough for the hosting?
    The back ups are taking so much space that I am really scared to have nothing left very quickly.My back ups are incremental by the way.

    Is it possible to have a database hosted on multiple HDS? for example 4 Terabytes space on 8 500HDs? Then what types of server do you need for this?

    Also, I am getting charged a lot for my remote backups, about £70 for 30GB, would it not be better to have a second server linked to the first one via a LAN 100Mpb cable instead of using the internet bandwidth and host all my backups of this?
    I do not understand why remote backups are so expensive with any host.


    A little bit of help would be more than appreciated,


    Regards,


    BamBam

  2. #2
    First off, you cannot have RAID-10 with 2HDDs, you need a minimum of 4, and second of all, if your backup space is managed by someone else and replicated, that price isn't all that high.

    Figure out how to split your application across several servers instead of building a monster server, it's always better to focus on scaling rather than speed.
    478east
    High Bandwidth Servers
    Custom Hosting Solutions

  3. #3
    Join Date
    Feb 2008
    Location
    California
    Posts
    315
    Quote Originally Posted by cristibighea View Post
    First off, you cannot have RAID-10 with 2HDDs, you need a minimum of 4, and second of all, if your backup space is managed by someone else and replicated, that price isn't all that high.

    Just wanted to point out here that you can have RAID-10 with two drives.

    We offer it on our two drive systems with a far layout. While write performance does take a hit, it gives read performance similar to RAID-0.
    Take 2 Hosting, Inc. - Hosting Done Right
    Fully automated setup - new servers in as little as 10 minutes
    http://www.take2hosting.com/

  4. #4
    OK Thanks for your help,


    Yes we have 2 HDs and not 4.
    Ok so what you mean really is dividing the database across different servers..ouch, that is going to be a hard job I bleieve.


    Thanks for your ehlp,


    Ben

  5. #5
    I should have said it's not very practical to have it setup, instead of impossible.

    I'm not sure what you understand by "database". It should be a MySQL database containing information about a video or an mp3 and on which server it sits, then the user would query your main website and be sent a link to the appropriate server hosting the file.
    478east
    High Bandwidth Servers
    Custom Hosting Solutions

  6. #6
    Join Date
    Mar 2009
    Posts
    2,222
    Presumably the media files do not change; so they do not need to be included in the regular backups. You just need to have two or more copies online, where they are available should a disaster occur.

    You yourself have to decide what kinds of disaster to cope with; but you could aim to cope with a hosting company going bust and leaving you no access to any of your data.

    The simplest way to deal with that for your application may be to have two or more servers, hosted by separate companies.

    When you get a new media file, you pick two servers to store it on; when a user is to download the file, one of the servers is picked at random.

    And you can increase throughput for popular files by storing them on more than two servers. Each server can support maybe half a dozen simultaneous downloads without slowdown, assuming users have 10mbps connections. A 5MB music file would take maybe 5 seconds to download, and each server could support maybe 4000 downloads per hour.

    If you don;t use multiple servers, you should work out how long it will take to restore 500GB of data. At 100mbps, that's about 10 million bytes a second, or 40GB per hour. So it would take over 12 hours to transmit 500GB of data across a 100mbps link.

  7. #7

    Thumbs up

    Thanks everyone for your great help,

    Tim thank you so much,I think you have answered my question perfectly!
    This is very clever, I have never thought about having the data going onto 2 servers at the same time and then get the data from only one server at the time from 2 different hosting company, this is really cool!
    I will forward this to my programmers as we have never worked on a website of this scale before.

    Thank you!

    BamBam

  8. #8
    Join Date
    Oct 2009
    Posts
    865
    Uhm, "RAID-10" on two disks is just RAID-1. A RAID-10 is a series of RAID-1 sets striped together in a RAID-0 configuration, but with 2 disks you only have the one RAID-1 set.

    If you want something that scales beyond one server, you might want to look into MogileFS or something similar.

  9. #9

    *

    Thanks Aearis I will look at this software later on :-)

    You are right about RAID 1 and RAID 10

  10. #10
    Join Date
    Feb 2008
    Location
    California
    Posts
    315
    Quote Originally Posted by aeris View Post
    Uhm, "RAID-10" on two disks is just RAID-1. A RAID-10 is a series of RAID-1 sets striped together in a RAID-0 configuration, but with 2 disks you only have the one RAID-1 set.

    That's not entirely accurate. RAID-1 is simple data mirroring. What's written to one disk is written to the other in the same manner.

    RAID-10 on two disks has the same logical affect as RAID-1 (meaning you get a copy of the data on each drive) but it can do this in few different ways. Depending on *how* the data is laid out, you get different performance.

    As mentioned above, we choose to do our RAID-10 with the "far layout". This means that the data is -- in essence -- laid out with two "RAID-0 stripes" on each disk. The data is written in a AB pattern in the first half of the disk and then a BA pattern on the second half of the disk.

    The affect of that particular layout, you have write speed that is reduced (because half the data written is in the first half of each disk and half the data is on the second half the disk) but you get read speeds that are similar to RAID-0 (because the data is striped between the two disks).

    Additionally, striping the data in this manner gives the system the opportunity to perform reads only on the outer portions of the drives where sequential read speeds are improved.

    For applications that are primarily read-based this can be a good storage choice.
    Take 2 Hosting, Inc. - Hosting Done Right
    Fully automated setup - new servers in as little as 10 minutes
    http://www.take2hosting.com/

  11. #11
    Join Date
    Oct 2009
    Posts
    865
    Quote Originally Posted by msherman View Post
    As mentioned above, we choose to do our RAID-10 with the "far layout". This means that the data is -- in essence -- laid out with two "RAID-0 stripes" on each disk. The data is written in a AB pattern in the first half of the disk and then a BA pattern on the second half of the disk.

    The affect of that particular layout, you have write speed that is reduced (because half the data written is in the first half of each disk and half the data is on the second half the disk) but you get read speeds that are similar to RAID-0 (because the data is striped between the two disks).

    Additionally, striping the data in this manner gives the system the opportunity to perform reads only on the outer portions of the drives where sequential read speeds are improved.
    That strikes me as a fairly weird way to partition data, and one I haven't encountered before. I suppose it could have a slight advantage in specific applications, provided that the storage controller is clever enough to optimize reads so they prefer the disk where the data in question is in the outer half.

    Although in my experience, read speeds in RAID-1 and RAID-0 are similar with a decent controller, it's write speeds that differ between the two.

  12. #12
    Join Date
    Aug 2009
    Posts
    38
    RAID 10 technically requires 2 drives at minimum, however, the industry standard is to use 4 because of the added speed benefits.

  13. #13
    Join Date
    Jun 2003
    Location
    The Netherlands
    Posts
    277
    You should probably be looking into cluster or cloud solutions.

    Here's something that might help you get started: http://gluster.com
    There are 10 kinds of people, those who understand binary and those who don't.

  14. #14
    thanks Talence I will look at your link tomorrow morning as it is too late for me now.

    I really appreciate your help,


    Ben

Similar Threads

  1. can afford a lot but need to have my servr secured
    By jeffbright in forum Dedicated Server
    Replies: 12
    Last Post: 12-16-2008, 05:04 PM
  2. Best servr management company
    By yadnum in forum Managed Hosting and Services
    Replies: 50
    Last Post: 10-07-2008, 02:12 AM
  3. servr management without CP
    By BookMarc in forum Managed Hosting and Services
    Replies: 13
    Last Post: 07-01-2008, 03:55 AM
  4. Moving Clients To Servr
    By ATOMIChost in forum Dedicated Server
    Replies: 2
    Last Post: 05-17-2003, 02:31 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •