Results 1 to 39 of 39
  1. #1
    Join Date
    Jan 2008
    Posts
    640

    Multi-Datacenter Enterprise HA Cloud Hosting. Is it possible?

    Hi,

    In a simple way, i just want to know if nowdays is it possible to really have a enterprise cloud server service whith High Availability based on at least 2 datacenters.

    In a simple way, i can only find hosting companies that can provide enterprise cloud servers in a very redundant way.. but all hosted on one single datacenter.

    My question is if their are hosting companies today that already have multi-datacenter HA cloud services?

    I know that this can be some kind of complex, because the infra-structure must allow real-time replication of data/information between different datacenters so a high speed private lan we think is at least needed.

    We also heard about some new commercial software that allows real-time replication of data and databases. Does anyone knows about this?

    So fell free to share you opinion and also if you know any hosting companies providing the service im looking for.

    Thanks

  2. #2
    Join Date
    Aug 2011
    Location
    Dub,Lon,Dal,Chi,NY,LA
    Posts
    1,838
    Spin up resources in two clouds, use Dyn.com dns failover to handle redirection.
    dediserve www.dediserve.com
    Leading provider of enterprise SSD cloud platforms with 15 clouds in 3 regions
    Dublin, London, Amsterdam, Vienna, Frankfurt, New York, Chicago, Dallas, Los Angeles, Toronto, Singapore, Jakarta, Hong Kong, Sydney

  3. #3
    Join Date
    Jan 2008
    Posts
    640
    Quote Originally Posted by dediserve View Post
    Spin up resources in two clouds, use Dyn.com dns failover to handle redirection.
    Hi dediserve,

    I would love it was so simple.

    But im specific talking about hosting an application that has many writes on database every second, so a real-time replication between databases and file system is needed. And i think the solution of using an external dns provider would not solve that issue, correct?

    The perfect scenario was for us to find a hosting provider who can provide 2 cloud in 2 different datacenters with real-time data/information replication between datacenters.

    Do you or anyone know any way to do this? Or a hosting company that is already providing this kind of service?

    Thanks

  4. #4
    Join Date
    May 2002
    Location
    Moscow
    Posts
    1,490
    Quote Originally Posted by pedrojose View Post
    Hi,

    So fell free to share you opinion and also if you know any hosting companies providing the service im looking for.

    Thanks
    Hello,

    Try contact to nac.net. They do not spread this info but i heard they offer redundant virtualization using two DC. Based on VMWare solution.
    Rustelekom LLC Dedicated server since 2002, RIPE NCC member, LIR, AS51168

  5. #5
    Join Date
    Jul 2012
    Posts
    52
    Nothing difficult in having a failover site in another DC. The main question is wheter the customer is willing to pay for it.

  6. #6
    Join Date
    Jan 2008
    Posts
    640
    Quote Originally Posted by Omiar View Post
    Nothing difficult in having a failover site in another DC. The main question is wheter the customer is willing to pay for it.
    We have the money. Can you tell us how or advice us on hosting companies that do this kind of service?

    Thanks

  7. #7
    Join Date
    Aug 2011
    Location
    Dub,Lon,Dal,Chi,NY,LA
    Posts
    1,838
    We have lots of custemers with mysql (and other dbs) being replicated in real time between our clouds.

    It's relatively simple to set up and monitor, then we recommend using something like Dyns dns failover and HA at DNS level options to either provide you with anycast geolocation or failover, depending on your requirements.
    dediserve www.dediserve.com
    Leading provider of enterprise SSD cloud platforms with 15 clouds in 3 regions
    Dublin, London, Amsterdam, Vienna, Frankfurt, New York, Chicago, Dallas, Los Angeles, Toronto, Singapore, Jakarta, Hong Kong, Sydney

  8. #8
    Join Date
    Jan 2011
    Location
    Canada
    Posts
    934
    You have to first define whether you need synchronous replication or whether asynchronous will suffice. That'll depend on the failover mode you want. Application performance leans towards asynchronous.

    Look up "Stretched Clusters" to get a better understanding of the costs and headaches and limitations involved with synchronous replication before you settle on it being a must-have.

  9. #9
    Join Date
    Aug 2011
    Location
    Dub,Lon,Dal,Chi,NY,LA
    Posts
    1,838
    One suggestion we've used with great success is to do replication in the same cloud, then replicate the replicant to a remote location. That way the primary DB can get replication out of the way without lag, and the replicated secondary can sync with the tertiary, where small delays are less impactful.
    dediserve www.dediserve.com
    Leading provider of enterprise SSD cloud platforms with 15 clouds in 3 regions
    Dublin, London, Amsterdam, Vienna, Frankfurt, New York, Chicago, Dallas, Los Angeles, Toronto, Singapore, Jakarta, Hong Kong, Sydney

  10. #10
    Join Date
    Jan 2011
    Location
    Canada
    Posts
    934
    Quote Originally Posted by dediserve View Post
    One suggestion we've used with great success is to do replication in the same cloud, then replicate the replicant to a remote location. That way the primary DB can get replication out of the way without lag, and the replicated secondary can sync with the tertiary, where small delays are less impactful.
    That's definitely the best way to do it.

    There's so much more flexibility if people loosen their "I must have real-time HA failover" if what they really meant was 15 minutes was good enough (with maybe 1 mins worth of data loss) if they didn't have to deal with a pager call.

  11. #11
    Quote Originally Posted by dediserve View Post
    One suggestion we've used with great success is to do replication in the same cloud, then replicate the replicant to a remote location. That way the primary DB can get replication out of the way without lag, and the replicated secondary can sync with the tertiary, where small delays are less impactful.
    really nice idea
    any script or something for monitoring, manual click and sync database option ..?


    https://web.easydns.com/our_nameservers.php
    this may help, if you planned to have data in another datacenter too, then adding this easydns feature will give improve in performance wise too.
    Simple and awesome
    Google

  12. #12
    Join Date
    Aug 2011
    Location
    Dub,Lon,Dal,Chi,NY,LA
    Posts
    1,838
    We run Zabbix in all our clouds, and use it to watch the replication lags (as well as lots of other metrics).

    If a machine falls out of sync for a set threshold an alert is raised.

    Keeps it nice and clean - mySQL has all the tools to do the job built in.
    dediserve www.dediserve.com
    Leading provider of enterprise SSD cloud platforms with 15 clouds in 3 regions
    Dublin, London, Amsterdam, Vienna, Frankfurt, New York, Chicago, Dallas, Los Angeles, Toronto, Singapore, Jakarta, Hong Kong, Sydney

  13. #13
    Join Date
    Jan 2008
    Posts
    640
    Quote Originally Posted by dediserve View Post
    One suggestion we've used with great success is to do replication in the same cloud, then replicate the replicant to a remote location. That way the primary DB can get replication out of the way without lag, and the replicated secondary can sync with the tertiary, where small delays are less impactful.
    Hi Dediserve,

    First i want to thanks you for all your advices and suggestions. Now, regarding the last idea you have given, it seems a good idea.. but if the case is datacenter / all platform cloud offline? This way it won't work.

    So or i did not understound correclty, or seems to me that the idea you have given only works in case of just the primary/main cloud server is offline. All the rest of the cloud and datacenter still have to keep up running.

    Correct? If yes, this is a good solution, but it won't give true and real time multi-datacenter high availability, because in case all cloud platform or main datacenter is down, no real time replicaiton is possible.

    Please tell me if im correct or if im missing something?

    Thanks

  14. #14
    Join Date
    Aug 2011
    Location
    Dub,Lon,Dal,Chi,NY,LA
    Posts
    1,838
    With the method I outlined you would have near real time replication to a remote cloud.

    So if cloud or data centre A had a total outage your DNS failover would sent traffic to you copy in cloud and data centre B.
    dediserve www.dediserve.com
    Leading provider of enterprise SSD cloud platforms with 15 clouds in 3 regions
    Dublin, London, Amsterdam, Vienna, Frankfurt, New York, Chicago, Dallas, Los Angeles, Toronto, Singapore, Jakarta, Hong Kong, Sydney

  15. #15
    @dediserve
    exactly
    @pedrojose
    https://web.easydns.com/feature_list.php

    above list contain fail over option, so if your cloud A DNS is down or not accessible means, all traffic will be routed to Cloud B without your intervention.
    Simple and awesome
    Google

  16. #16
    Quote Originally Posted by pedrojose View Post
    But im specific talking about hosting an application that has many writes on database every second, so a real-time replication between databases and file system is needed. And i think the solution of using an external dns provider would not solve that issue, correct?
    dediserve provided you a reasonable solution. maintaining the mysql clustering in such a manner is not a simple set and forget scenario - but, with the availability of sysadmins, it can be accomplished and is certainly a reasonable/cost effective solution

    having said this, to your original question - you are asking infrastructure to do something its not meant to do. the "cloud" cant really enable high availability of your application. it can certainly help, but, when you are speaking real time replication, across distance of very heavy write databases - its a real issue.

    The perfect scenario was for us to find a hosting provider who can provide 2 cloud in 2 different datacenters with real-time data/information replication between datacenters.
    this just doesnt exist. the ideal scenario, and the way this is typically done - is to fire up instances in multiple facilities, and write an application, which is designed to replicate in the manner you are describing. DNS is always an issue - so, any strong anycast DNS provider can handle that part for you (dnsmadeeasy or whoever you prefer). the key point however is building your application stack on a platform that can best accommodate this and purposefully built for this. for example, try and avoid mysql - its replication ability is getting better, but, its not really there yet. maybe look to build your application on something like http://www.mongodb.org/

    im presuming that mirroring the app data itself will be pretty routine, especially if it doesnt change often. php or any modern language will do the trick. the real complexity comes with the backend database, especially one, which as you described, is very write heavy
    Last edited by cartika-andrew; 11-28-2013 at 09:58 PM.

  17. #17
    I don't know if i understood dediserver when he says:

    "do replication in the same cloud, then replicate the replicant to a remote location. That way the primary DB can get replication out of the way without lag, and the replicated secondary can sync with the tertiary, where small delays are less impactful."

    If the primary DB has a heavy usage and has to be replicated remotely to a diferent data center in other continent, how do you sync the data in real time?

  18. #18
    Join Date
    Aug 2011
    Location
    Dub,Lon,Dal,Chi,NY,LA
    Posts
    1,838
    My proposed solution was to have the secondary server in the same location as the primary database. Therefore linked over private LAN, and running in real time replication.

    The Secondary then replicates with a tertiary server, which is in the remote location.

    In our experience this allows occasional lag to be handled without any impact to the production systems, and gives a local backup should it be needed also.
    dediserve www.dediserve.com
    Leading provider of enterprise SSD cloud platforms with 15 clouds in 3 regions
    Dublin, London, Amsterdam, Vienna, Frankfurt, New York, Chicago, Dallas, Los Angeles, Toronto, Singapore, Jakarta, Hong Kong, Sydney

  19. #19
    @digitalvalley
    you wrongly understood buddy,

    he told, we have to replicate/sync the primary db (database A) in same datacenter in same hosting company to another db, say database B. Then database B is again sync with Database C which is placed in another datacenter.

    Database A - in Datacenter 1
    Database B - in Datacenter 1, and same hosting as database A, so it use LAN for writing in Database B (LAN ping is less than micro second or 1 or 2 millisecond) so even the write load is high in Database A, it is easier to copy in same datacenter

    Database C - in another continent, another datacenter (which use OpenVPN) or someother way, so it connect the remote db (database B) as fast as possible.


    hope you understood
    Simple and awesome
    Google

  20. #20
    Understood now, however between database a/b and C there is no real time sync, there is always a lag, if datancenter A goes down completely, database C is not updated. The first question said that they wanted to have "real-time" replication of their databases in diferente data centers.

    The solution from dediserver is very good but there is a lag with the remote C server. Since the first person wanted to have real HA between the two diferente dc, the solution is close but is not "real-time" because "real-time" is very complex to obtain in diffrent data centers because of the network lag

  21. #21
    Join Date
    Aug 2011
    Location
    Dub,Lon,Dal,Chi,NY,LA
    Posts
    1,838
    The lag is, in real terms, very small.

    We have a solution exactly like this for a company with very write heavy DB (they use FusionIO to handle it) and the lag from DB B to DB C never exceeds 2 minutes. Most of the time it's effectively zero.
    dediserve www.dediserve.com
    Leading provider of enterprise SSD cloud platforms with 15 clouds in 3 regions
    Dublin, London, Amsterdam, Vienna, Frankfurt, New York, Chicago, Dallas, Los Angeles, Toronto, Singapore, Jakarta, Hong Kong, Sydney

  22. #22
    @digitalvalley
    real time is possible, because we're not going to delete the old db and import new db..

    we just add the extra tables and content to database, that is same like pause the mp3 song in ipod, then we click play it start from where we clicked pause..so yeah possible..


    100 % real time is possible if database if offloaded to Amzon rds like external app
    http://aws.amazon.com/rds/

    that is we update read/write to cloud (100% uptime possible), because in amazon cloud

    we can use any number of static content server in any datacenter, nothing to worry about
    Simple and awesome
    Google

  23. #23
    Join Date
    Jan 2011
    Location
    Canada
    Posts
    934
    Quote Originally Posted by dediserve View Post
    The lag is, in real terms, very small.

    We have a solution exactly like this for a company with very write heavy DB (they use FusionIO to handle it) and the lag from DB B to DB C never exceeds 2 minutes. Most of the time it's effectively zero.
    Heh: most of the time.
    @digitalvalley

    Just throwing it out there, but if your write ops is so low that you're effectively getting zero async lag, AND consistency is important AND at the same time you don't have a separate audit system setup for verifying transactions, then you need to look at something like XtraDB or MySQL semi-synch. Either of which has relatively poor performance over WAN but at least gives you the consistency aspect.
    Last edited by tchen; 11-29-2013 at 12:06 PM.

  24. #24
    Join Date
    Jan 2011
    Location
    Canada
    Posts
    934
    Quote Originally Posted by feastboy View Post
    @digitalvalley
    100 % real time is possible if database if offloaded to Amzon rds like external app
    http://aws.amazon.com/rds/

    that is we update read/write to cloud (100% uptime possible), because in amazon cloud

    we can use any number of static content server in any datacenter, nothing to worry about
    Except that the multi-AZ RDS only protects against DC related problems like power and network. Not to mention they're only separated by less than 150 miles. The backbone connecting AZs has been saturated in the past, bringing down an entire region. It's good - better than nothing - but it might not cover the worry-cases that people seeking multi-DC DR are seeking to address.

  25. #25
    Join Date
    Jun 2011
    Location
    Toronto, Canada
    Posts
    47
    Hello All

    There is a way to replicate the file system between two datacenters by the means of replication on the storage level (many storage providers do have such option), but the secondary VM has to be kept offline and when necessary turn it on,

    All OS level as well as Databases will be synced in real time
    Cirrus Tech. Ltd. - A Canadian Cloud hosting Company, Since 1999
    www.cirrushosting.com

  26. #26
    Quote Originally Posted by feastboy View Post

    100 % real time is possible if database if offloaded to Amzon rds like external app
    http://aws.amazon.com/rds/
    the solution recommended by dediserve will actually work out better on the whole. just because Amazon says they provide tools for mysql replication - it doesnt mean their "cloud" enables anything. Mysql replication is a built in feature of mysql

    http://dev.mysql.com/doc/refman/5.0/...ion-howto.html

    as I said in my previous post - you are looking for infrastructure (ie the cloud) to enable HA in your application. As long as you go this route, you will not be able to achieve what you are trying to achieve.

    as someone else mentioned, amazon has their facilities very close to each other, which helps.. but, 1) do you really want your locations that close to each other? and 2) they have had congestion issues which can and will cause delays in replication much larger then the 2 mins dediserve recommended.

    this is further complicated if you want to actually replicate this over ever larger distances, over the WAN, etc..

    the only real solution is to build your application on a platform which natively supports the features you want

    as long as you are expecting the "cloud" to do this for you, all magically in the background, you will not get the results you want

  27. #27
    Join Date
    Aug 2011
    Location
    Dub,Lon,Dal,Chi,NY,LA
    Posts
    1,838
    ^^^^^^ What Andrew Said.

    "Cloud" won't magically provide any HA unless you design for it in the first place.
    dediserve www.dediserve.com
    Leading provider of enterprise SSD cloud platforms with 15 clouds in 3 regions
    Dublin, London, Amsterdam, Vienna, Frankfurt, New York, Chicago, Dallas, Los Angeles, Toronto, Singapore, Jakarta, Hong Kong, Sydney

  28. #28
    Join Date
    Jan 2008
    Posts
    640
    Quote Originally Posted by CirrusTech View Post
    Hello All

    There is a way to replicate the file system between two datacenters by the means of replication on the storage level (many storage providers do have such option), but the secondary VM has to be kept offline and when necessary turn it on,

    All OS level as well as Databases will be synced in real time
    Hi CirrusTech,

    Thanks for your idea/advice. It seems simple from your point of view. Can you explain it better

    To all other that have talked here, i thank you as well for all the advices provided... but i still don't see an effective and reliable multi-datacenter HA solution that can allow real-time replication specially for the database.

    We first think to go with a cloud provider. But and if we go with dedicated servers, will the result be also the same? Even this way cloud is better that going with dedicated servers?

    Also if anyone has any idea or solution, fell free to share.

    Thanks

  29. #29
    Join Date
    Jan 2011
    Location
    Canada
    Posts
    934
    Quote Originally Posted by pedrojose View Post
    To all other that have talked here, i thank you as well for all the advices provided... but i still don't see an effective and reliable multi-datacenter HA solution that can allow real-time replication specially for the database.
    As Andrew said, this is typically an application level thing. It's all about performance-consistency tradeoffs which requires knowledge about the business. I highly recommend listening to this presentation:

    https://www.usenix.org/conference/li...ers-and-clouds

    As a cheat sheet, and I apologize for throwing the terms at you earlier without explanation:

    HA: High availability.

    DR: Disaster recovery.

    RTO: Recovery Time Objective. The desired time it takes to flip the switch to failover to the DR site.

    RPO: Recovery Point Objective. Acceptable rollback point in case of disaster.

    Synchronous Replication: Remote server needs to acknowledge before a database transaction is committed. Throughput is affected by latency. However, both A/B sites will have the exact data at any given time, hence perfect RPO.

    Asynchronous Replication: Relaxed acknowledgement. Typical MySQL Master(A)-Slave(B) setup where past commits are streamed and replayed on slave. Not latency bound. A is ahead of B as determined by network bandwidth. This gap needs to be kept below the RPO and is a bandwidth issue.

    Block Replication: Disk bytes are replicated from A to B. Typically synchronous and used within a cluster to avoid downtime with individual nodes. Can work generically for both database and files.

    Stretch-Cluster: A short-range multi-DC setup that employs synchronous replication at either the block or database levels. Typically limited to 300km due to latency issues. Used for HA and limited DR.

    HA Tierd-Application: A pair of load-balancers, followed by web servers and replicated storage. Typically designed to be self-healing and with very short RTO on the order of minutes or less depending on complexity of network setup.

    Graceful Degradation: A method of making a site's features work with more limited resources. Useful in the case of failover where capacity at the DR site is a fraction of the main site.

    Eventual Consistency: A database term which arises because of asynchronous replication lag. Accessing a node that isn't the one being written to may not produce the expected data. Eventually it will when caught up.

    Galera/XtraDB: A highly-available performant synchronous MySQL cluster that can tolerably span WANs.

    MongoDB: A highly-available NoSQL server designed for high-throughput writes. Can have mixed synchronous/asynchronous writes. Tunable for multi-DC awareness.

    Couchbase: A highly-available NoSQL server designed for high-throughput writes. Has asynchronous multi-DC replication.


    Where are you on the RTO-RPO spectrum? RTO can be easier sometimes if the DR site is a cloud since you can typically provision faster. And where are you on the legacy app spectrum? Is this something that was designed to be a single-server deploy?

  30. #30
    We are building a high availability cloud platform across 3 failover datacenters at the moment. The virtual servers are synchronized in real time. If one datacenter fails the servers are transfered in another datacenter and started automatically within minutes or seconds. There is no need to change IP addresses or anything like that. The HA system is transparent to the user.

    The datacenters are spread accross one city and interconnected with fiber cables to ensure performance. Our service will be available in February 2014.

    Best
    Tom

  31. #31
    Join Date
    Jan 2008
    Posts
    640
    Quote Originally Posted by tchen View Post

    - Synchronous Replication: Remote server needs to acknowledge before a database transaction is committed. Throughput is affected by latency. However, both A/B sites will have the exact data at any given time, hence perfect RPO.

    - Stretch-Cluster: A short-range multi-DC setup that employs synchronous replication at either the block or database levels. Typically limited to 300km due to latency issues. Used for HA and limited DR.

    - HA Tierd-Application: A pair of load-balancers, followed by web servers and replicated storage. Typically designed to be self-healing and with very short RTO on the order of minutes or less depending on complexity of network setup.

    - Galera/XtraDB: A highly-available performant synchronous MySQL cluster that can tolerably span WANs.
    Hi @tchen,

    Thanks for your detailed explanation. This really help me understand things better.

    I have quoted above the 4 main points that we think are more important for our project. The "Synchronous Replication" really seems to be the best option.

    So using "Synchronous Replication" on servers hosted on 2 different Datacenters located no more than 300Km from each other using a private 1Gbps private LAN seems perfect for our needs.

    With this kind of setup i supose our application will be a little bit slower in terms of performance because of the "Synchronous Replication" process that has to run every time there is a write on the Database, but will ensure 100% Perfect an Real Time Multi Datacenter replication.

    So in a simple way will make our application a little bit slower but will ensure 100% Perfect an Real Time Multi Datacenter replication, correct?

    Any advices or experiences using this kind of setup? Is this the right way or someone has a better idea?

    Thanks

  32. #32
    Join Date
    Jul 2006
    Location
    Lake Zurich, IL
    Posts
    282
    Quote Originally Posted by pedrojose View Post
    So using "Synchronous Replication" on servers hosted on 2 different Datacenters located no more than 300Km from each other using a private 1Gbps private LAN seems perfect for our needs.

    With this kind of setup i supose our application will be a little bit slower in terms of performance because of the "Synchronous Replication" process that has to run every time there is a write on the Database, but will ensure 100% Perfect an Real Time Multi Datacenter replication.
    The 2-phase commit for synchronous replication will be pretty slow with 300km between sites "per transmission". This distance will add a minimum of 2ms of latency (back and forth), but probably more like 4ms to 6ms with overhead. At 4ms, that is 250 round-trips per second - not super fast.

    Eric
    Genesis Hosting Solutions, LLC
    http://www.genesishosting.com/
    Instant VMware vSphere Cloud Environments
    Unlimited virtual machines within your purchased resources!

  33. #33
    Join Date
    Aug 2011
    Location
    Dub,Lon,Dal,Chi,NY,LA
    Posts
    1,838
    Bear in mind that you will also hold up the primary DB server, as it waits relative 'eons' for the syn (relative to the local flash or whatever it runs on) (4ms versus microseconds)
    dediserve www.dediserve.com
    Leading provider of enterprise SSD cloud platforms with 15 clouds in 3 regions
    Dublin, London, Amsterdam, Vienna, Frankfurt, New York, Chicago, Dallas, Los Angeles, Toronto, Singapore, Jakarta, Hong Kong, Sydney

  34. #34
    Join Date
    Jun 2011
    Location
    Toronto, Canada
    Posts
    47
    Quote Originally Posted by pedrojose View Post
    Hi CirrusTech,

    Thanks for your idea/advice. It seems simple from your point of view. Can you explain it better

    To all other that have talked here, i thank you as well for all the advices provided... but i still don't see an effective and reliable multi-datacenter HA solution that can allow real-time replication specially for the database.

    We first think to go with a cloud provider. But and if we go with dedicated servers, will the result be also the same? Even this way cloud is better that going with dedicated servers?

    Also if anyone has any idea or solution, fell free to share.

    Thanks
    Well it is not going to be easy to explain here all the details, but there are SAN/NAS solutions which allow you to synchronize all the Virtual Machine Data in block level and in secondary locations,

    I am sure some providers will be able to offer such based on your requirements under their Private Cloud offering.
    Cirrus Tech. Ltd. - A Canadian Cloud hosting Company, Since 1999
    www.cirrushosting.com

  35. #35
    Join Date
    Jan 2011
    Location
    Canada
    Posts
    934
    Quote Originally Posted by pedrojose View Post
    Hi @tchen,

    Thanks for your detailed explanation. This really help me understand things better.

    I have quoted above the 4 main points that we think are more important for our project. The "Synchronous Replication" really seems to be the best option.

    So using "Synchronous Replication" on servers hosted on 2 different Datacenters located no more than 300Km from each other using a private 1Gbps private LAN seems perfect for our needs.

    With this kind of setup i supose our application will be a little bit slower in terms of performance because of the "Synchronous Replication" process that has to run every time there is a write on the Database, but will ensure 100% Perfect an Real Time Multi Datacenter replication.

    So in a simple way will make our application a little bit slower but will ensure 100% Perfect an Real Time Multi Datacenter replication, correct?

    Any advices or experiences using this kind of setup? Is this the right way or someone has a better idea?

    Thanks
    Just a clarification, VMware only suggests < 100km for stretched clusters that replicate at the file block level. 300km is more of the maximum for something like IBM SAN and only under specific tolerance levels. Even they don't suggest going over 150km.

    If we restrict replication to only database, it's a little more tolerant but still suffers a huge penalty. There was a performance test of XtraDB between east and west AWS datacenters which shows a transactions per second of about 1/4 that of a local master-slave setup.

    I should also amend my post by saying that even given the synchronous nature, there are rare edge cases that will cause you to have consistency issues that need to be remedied during failover. It's actually not 'perfect' so please don't rely on it as your only guarantee if this is a mission critical system of record. Those situations require auditing and other distributed transaction architectures.

  36. #36
    VMware solution is def. the right path.

  37. #37
    Join Date
    Jun 2013
    Posts
    36
    So, two site HA with synchronous replication is definitely possible. We're doing it with a number of customers. We use a VXLAN wire which operates inside two datacentres, so from the customer's perspective its a single layer 2 segment. Machines in DC A can reach machines in DC B directly and with <1ms latency.

    We deploy a pair of firewalls / balancers for the customer, one in each site. They have 2 or more webservers, split over sites. two databases servers, one in each site. To date we've done this with MS SQL as customers were windows, but no issue doing the same with other databases or OSs.

    In the event of complete datacentre outage, all of the IP/routing is automatically switched over by the network stack. The primary firewall and balancer will go offline, so the secondary will grab its virtual IPs. In the case of SQL mirroring, the database will be failed over and bingo, everything works.

    There's nothing particularly difficult about this. All of the networking (VXLAN, firewalling, balancing etc) is provided via VMware products. We have around 65Km between datacentres over 2 x 10Gb waves, so latency and synchronous replication/mirroring etc is no problem.

    We've used synchronous mirroring because up to know the customers doing this cannot tolerate *any* dataloss whatsoever in the event of an outage. However if the distance was greater, or the level of writes in the database high then async could be used with a tiny delay. And naturally, the idea of a premium datacentre going dark is a bit hard to see how.

    HTH.

  38. #38
    This is something that can be done - but it is usually very expensive and limited by distance (speed of light)and even more expensive if you want to try and remove the distance limitations. Things like EMC Recoverpoint with Phoenix appliances, SRM, and SRDF are all ways to accomplish what you're looking for. Expect to pay 60-100k per month for a high performance solution that is geographically redundant that has high levels of performance.

    Sungard is probably the market-leader when it comes to talking about DR solutions, that's probably where I'd look if you're wanting to really get into a hardcore DR/Business Continuity solution.
    ZFS Storage Blog - http://www.zfsbuild.com/

  39. #39
    Join Date
    Dec 2013
    Location
    Europe and US
    Posts
    17
    What an option could be is to setup a private bittorrent network to replicate the data between the several locations.
    Then you can synchronize the data in blocks.
    Or the solution of 3par could help you out as well, but is very expensive.

Similar Threads

  1. Replies: 0
    Last Post: 07-12-2013, 09:38 AM
  2. ENTERPRISE || Cloud Computing || HIGH-END PREMIUM CLOUD HOSTING!
    By ClubVPS-Yohay in forum Dedicated Hosting Offers
    Replies: 0
    Last Post: 07-03-2012, 04:46 AM
  3. Replies: 0
    Last Post: 02-24-2009, 11:18 AM
  4. Replies: 0
    Last Post: 12-30-2008, 02:46 PM
  5. Replies: 0
    Last Post: 12-30-2008, 02:35 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •