What is the best way to configure the following:

server1 host1.com
host2.com
host3.com

server2 host1.com
host2.com
host3.com

Where both web servers actively serve all three host domains?

I've some configuration ideas, but am looking for input.

Thanks,

-t

Im not 100% sure but how about this:

1) are the sites Static data or are they webmail, sql, submission forms, e.g. dynamic data?

if they are static then you should be able to set up a DNS Pool and have people get automatically distributed amongst the servers.

if it is dynamic you will probably need another server to host the dynamic data and have the front ends distributed amongst the two servers.

What are the mechanics of setting up a DNS Pool?

We can assume static sites for the sake of this discussion.

-t

Aloha,
are you talking about clustering them ??
where the 2 machines runs as one machine ?

or are you trying to run 2 machines seperate with the same domain on them ???
not sure why you would want to do the second one

the best choice is to run your two machines as one machine using CLUSTERING this will provide a way for your machines if one fails to have the other machine keep running for you.
it can also handle load better as the requests can balance between the two machines
so maybe a little more info of what you are trying to do.

have redundancy ?
a spare backup ?

the fun of figuring stuff out ;)

Not interested in clustering, as the model I would like to use applies to "n" servers and "n" domains.

Yes, I would like to have the domains resolve to multiple hosts.

DNS Round Robin looks promising, however it makes me a little nervous without having done this already.

I guess I'm wondering what the state of the art here is. Cisco sells a product called the LocalDirector that looks like it does this, but I don't know why I couldn't just do this with DNS.

-t

Are you planing on using different third level domains for the hosts?

Server1
www.host1.com
sub.host2.com
other.host3.com

Server2
support.host1.com
www.host2.com
sub.host3.com


Or are you trying to use the same third level domains?

Server1
www.host1.com
www.host2.com
www.host3.com

Server2
www.host1.com
www.host2.com
www.host3.com


If you just need to split different subs between the two servers then DNS pooling will work for you. Actually I would just set server1 up as the primary and server2 as the secondary, create all records on server1 and then just distribute the A records to the different IPs on the 2 boxes.


On the other hand, if you want to use the same third level domains for the same hosts on both machines you WILL NEED CLUSTERING!!


;)

DNS Round Robin will do what I want.

I'll post the final configuration at some point, but I've tested it with three servers on a private network and it works, so I'm happy for the time being.

My next experiments will be with Dynamic DNS, since I now need to be able to remove a server from the pool of servers should I lose the hardware, otherwise the Round Robin access to that server will fail. I'd like to be able to do that dynamically - potentially having a failed server automatically removed from the DNS Round Robin by fact it is not running...

The DNS Round Robin goes from A to B to C to A to B to C, etc. If B is dead, every 3rd request fails. Not sure yet how to automatically remove myself if I fail, but I'll come up with something I'm sure :D

-t

Cisco LocalDirector deals with fault tolerance..... of course there is a cheaper alternative, http://www.linuxvirtualserver.org/

Aloha,
yeah round robin is basically clustering it just does not use load balancing rather then sensing what server is under load and sending it just gos down the line 1 then 2 then 3
the down side to this (someone correct me if I am wrong) is that if one of the boxes should fail then round robin will send that user to the failed box (and the next 3rd user and so on giving you every third person a dead box without knowing it failed and the user will get a dead box.
where as load balancing it will first check the box then redistribute if needed
also say you have 3 boxes and 30 users connected just by chance in round robin if your users for a freak reason are all taxing the box #2 then the load will go very high on that box
using load balancing can be better ???

best of luck
ya just trying to make a system that will be more failproof or ???
just curious ;)

Originally posted by JTY
Cisco LocalDirector deals with fault tolerance..... of course there is a cheaper alternative, http://www.linuxvirtualserver.org/

Very nice. I'll have to study this one more.

I've had a demonstration of the Cisco product. They really want to sell it to you with all their content delivery tools as well. Very expensive too.

-t

Originally posted by Honu
Aloha,
yeah round robin is basically clustering it just does not use load balancing rather then sensing what server is under load and sending it just gos down the line 1 then 2 then 3
the down side to this (someone correct me if I am wrong) is that if one of the boxes should fail then round robin will send that user to the failed box (and the next 3rd user and so on giving you every third person a dead box without knowing it failed and the user will get a dead box../quote]

This is correct. You need to sense that the server is unresponsive or overloaded and remove it from the DNS pool. An automated method for doing this would be better than manually doing so of course.

[quote]
where as load balancing it will first check the box then redistribute if needed
also say you have 3 boxes and 30 users connected just by chance in round robin if your users for a freak reason are all taxing the box #2 then the load will go very high on that box
using load balancing can be better ???

best of luck
ya just trying to make a system that will be more failproof or ???
just curious ;)

I'm not real pleased with any of the load balancing/clustering options that I've looked at so far. I'm back to the lowest level - the DNS - and moving forward slowly again. It seems that many people have high $$$$ solutions they claim solve this problem. I'm looking to do it on the cheap. :)

If I can solve 90% of the issues, I'll save the 10% that remain for sites that seek to be highly available by design.

And yes, I'm looking at a more failproof configuration for web hosting, as well as one that scales more easily for the customer.

I've an interesting business model under development that may change the way some of you do business in the future as well.
:D

-t

Hello

We've been using roundrobin DNS hosting since our inception last year while we don't advertise it but it is proven to be effective and it is a low cost solution.

75% of sites we host are on dual data centers w/ 2 servers running each own dns ns1. -->data center1 ns2--> data center 2

25% of sites we host are on quad data centers w/4 servers running on each site ns1-->dc1 ns2-->dns1-->dc3 and so on...

it's simple and while you really cannot offer 100% uptime but at least you can offer a HighAvailability solution on a different level. if you need more info on how we did ours , email me and i will forward your email to my older brother.

note: clustering and load balancing is still a better solution.


cheers,
elijah john :)

Heh, I completely forgot about this post!

The DNS Pool i was talking about is indeed DNS Round Robin, I juts like to confuse people and name things differently :-)

Looks like you have set it up anyways, keep us updated good luck!

Originally posted by thewitt
The DNS Round Robin goes from A to B to C to A to B to C, etc. If B is dead, every 3rd request fails. Not sure yet how to automatically remove myself if I fail, but I'll come up with something I'm sure :D


I'm not entirely certain about this, but I think that you could do this by making the 3 servers serve DNS for those domains inconsistantly. That is, have each server respond with its own IP address when a DNS query comes in. In addition, set the DNS timeout to be relatively short (30 seconds max).

That way, if one server is down, it won't respond to the DNS query, and a different machine will be queried, and people will be sent to whichever machine was randomly picked to answer the DNS query. By setting the TTL quite short on the DNS you make sure that DNS caches won't keep sending people to a downed server, at the cost of an increase in DNS traffic.

As far as I can tell this is the only solution which doesn't require a separate traffic director/load balancer/reverse proxy/whatever the name of the month is.

For simple web sites that are primarily HTML and graphics, round-robin DNS is enough...

Alas, most sites these days aren't so simple. Does your server have any sessions (shopping carts, etc.?).

If you have a database, you have to make a lot of key decisions on how you intend to deal with failure. Even if your database (DBMS) supports replication, you have to consider when that isn't working right either.

Is there data (database) involved that the servers must have common knowledge of? How much data loss (inconsistency) is acceptable?

I am working on a project to define the various ways of doing web site redundancy, the name is MISTUS - "Multiple Internet Site Technology Using Software" -- my intention is to make (hard disk) RAID style analogies to describe the various ways that different you can solve the problems. Mirrored servers, client-based load balancing (DNS) vs. server based (clustering), etc.

I'm still in the early stages, but I am hoping to make this a public project. http://www.mistus.org/ will be the future home...

As long as the database is served by a specific server and not at localhost, using a database in this configuration is no problem.

Why do you assume that there will be data loss?

-t

Are you suggesting you have multiple web servers with only a SINGLE database server?

That leaves a single point of failure... what if your data corrupts, hardware problem, or the database software just stops?

For most database-driven sites, like an on-line store, the site is useless without the database.


The data loss question
==================
I'm saying that if you have multiple database servers, there are trade-offs between how soon you replicate data and so forth.

Simple way of describing it: If you replicate only every 5 minutes, then if a database server died -- you have 5 minutes worth of potential data loss.... assuming that server never came back to life.