If i had a UK box and a USA box. Would i be able to have one as primary DNS and one as secondary?
if so, how will this be better?
does dns take line of sight? like closest dns sevrer and it uses that?

Regards

Craig

Yes, you could.

Would it be better? Possibly. Having a working primary & secondary DNS will make it so if your primary DNS box goes down your DNS will still work. It won't effect performance, though. DNS lookups are automatically sent to the primary server. Only if the primary server is down will the secondary server be queried.

Originally posted by Planet Z
It won't effect performance, though. DNS lookups are automatically sent to the primary server. Only if the primary server is down will the secondary server be queried.

That is _not_ correct.

Performance wise it is a very good idea to have geographically distant DNS servers. Clients will send DNS requests to the DNS server that responds fastest (think of pinging both servers, and choosing the one with the lowest ping time).

--
Jens Kristian Søgaard, Mermaid Consulting I/S,
jens@mermaidconsulting.dk,
http://www.mermaidconsulting.com/
http://www.raqsupport.net/

Originally posted by jks


That is _not_ correct.

Performance wise it is a very good idea to have geographically distant DNS servers. Clients will send DNS requests to the DNS server that responds fastest (think of pinging both servers, and choosing the one with the lowest ping time).

--
Jens Kristian Søgaard, Mermaid Consulting I/S,
jens@mermaidconsulting.dk,
http://www.mermaidconsulting.com/
http://www.raqsupport.net/

Well, that's not entirely true either....though it should be.

It really depends on your upstream provider and who you list in your resolver files. More often then not, nameservers query first-listed as opposed to best-response.

As to the original question, geographically diverse servers are good because if your network dies then you have secondary DNS which can direct your mail to an offsite backup MX host.

-davidu

Originally posted by DavidU

It really depends on your upstream provider and who you list in your resolver files. More often then not, nameservers query first-listed as opposed to best-response.

Hmm, which nameservers query first-listed as opposed to best-response?

And again, if you (=the authoritative host for the domain) use Bind - then NS records will often be handed out randomly (i.e. the order is not the same each time). That will give some "load balancing" too -- i.e. it's not the primary source that is asked first.


--
Jens Kristian Søgaard, Mermaid Consulting I/S,
jens@mermaidconsulting.dk,
http://www.mermaidconsulting.com/

Originally posted by jks
[B]

Hmm, which nameservers query first-listed as opposed to best-response?

And again, if you (=the authoritative host for the domain) use Bind - then NS records will often be handed out randomly (i.e. the order is not the same each time). That will give some "load balancing" too -- i.e. it's not the primary source that is asked first.


http://cr.yp.to/djbdns.html (http://cr.yp.to/djbdns.html) -- Great Namserver...

as a response to your post I was thinking. all the root com/net/org nameserver run BIND however looking at my own logs, ns1.everydns.net gets hit about 95% of the time compared to ns2.everydns.net -- That wouldn't be right if the top level nameservers handed out a random select of ns1 or ns2 for a given delegated domain?

I know DJBDNS supports random resolving returns for A and CNAMES but not MX or NS (or others). It was my understanding that bind was the same. Are you thinking of A and CNAME records for authoritative name servers? The top level nameserver which only hand out NS records I do not believe are "random" but instead are sorted in order.

-davidu

Well, it may be a great nameserver -- but it isn't RFC compliant. The RFC demands that it must query best-response and not first-listed. You're right that the toplevel servers probably deliver NS records in a fixed fashion - but the querying server should not use them in that way.

As to your empirical testing - is there anything else that could lead to that 95% vs. 5% ?
(For example: Do they have the same internet connection? - Is one server faster than the other? etc.)


--
Jens Kristian Søgaard, Mermaid Consulting I/S,
jens@mermaidconsulting.dk,
http://www.mermaidconsulting.com/

Originally posted by jks
[B]Well, it may be a great nameserver -- but it isn't RFC compliant. The RFC demands that it must query best-response and not first-listed. You're right that the toplevel servers probably deliver NS records in a fixed fashion - but the querying server should not use them in that way.

As to your empirical testing - is there anything else that could lead to that 95% vs. 5% ?
(For example: Do they have the same internet connection? - Is one server faster than the other? etc.)


No, it's bind which isn't always RFC compliant which is funny since those are the clowns who write most of the DNS RFC's (ISC Coroporation) DJBDNS isn't compliant in other ways, but it's just cuz DJB feels the ISC sucks and he is better then them. (I make no comment here... heh)

Anyways, yeah, both boxes are the same network. I guess when we get our third nameserver up in Northern California we'll have a better picture.

-davidu

Originally posted by DavidU

No, it's bind which isn't always RFC compliant which is

Ehm, I think not. RFC 1035 states quite clearly what must happen:

"the desired behavior is that the resolver transmit queries to name servers in a way that maximizes the probability that the request is answered, minimizes the time that the request takes, and avoids excessive transmissions."

And further:

"To complete initialization of SLIST, the resolver attaches whatever history information it has to the each address in SLIST. This will usually consist of some sort of weighted averages for the response time of the address, and the batting average of the address (i.e., how often the address responded at all to the request). Note that this information should be kept on a per address basis, rather than on a per name server basis, because the response time and batting average of a particular server may vary considerably from address to address. Note also that this information is actually specific to a resolver address / server address pair, so a resolver with multiple addresses may wish to
keep separate histories for each of its addresses. Part of this step must deal with addresses which have no such history; in this case an expected round trip time of 5-10 seconds should be the worst case, with lower estimates for the same local network, etc."

funny since those are the clowns who write most of the DNS RFC's (ISC Coroporation)

I have on several occasions emailed with the programmers of Bind. I don't find it reasonable to call them "clowns".


DJBDNS isn't compliant in other ways, but it's just cuz DJB feels the ISC sucks and he is better then them. (I make no comment here... heh)

I don't know if you know this.. but some people don't use DJBs software simply because it's written by him.

Anyways, yeah, both boxes are the same network. I guess when we get our third nameserver up in Northern California we'll have a better picture.

I run several (10+) loaded DNS servers - and it's certainly not the same percentage I'm getting.

--
Jens Kristian Søgaard, Mermaid Consulting I/S,
jens@mermaidconsulting.dk,
http://www.mermaidconsulting.com/

Originally posted by jks
[B]

I don't know if you know this.. but some people don't use DJBs software simply because it's written by him.



I know, and that's why I make no comment. Personally, I don't judge a person's code based on their atitude. the license thing is totally different.

As to why I don't run bind, well it's just a security nightmare and a game of catch-up I don't want to play.

-davidu

Originally posted by DavidU


As to why I don't run bind, well it's just a security nightmare and a game of catch-up I don't want to play.


For how long have you been a DNS administrator?

If you look back and judge a few years of Bind - you won't see many exploits...

I run my own version of Bind9 in a chrooted jail - I don't see any problems with that...

--
Jens Kristian Søgaard, Mermaid Consulting I/S,
jens@mermaidconsulting.dk,
http://www.mermaidconsulting.com/

Originally posted by jks


For how long have you been a DNS administrator?

If you look back and judge a few years of Bind - you won't see many exploits...

I run my own version of Bind9 in a chrooted jail - I don't see any problems with that...

--
Jens Kristian Søgaard, Mermaid Consulting I/S,
jens@mermaidconsulting.dk,
http://www.mermaidconsulting.com/

Jens, you also know you aren't the typical DNS admin. Half the cluster****s out there who call themselves admins don't know what a chroot()ed environment is let alone how to create one. I'll admit Bind9 is a significant improvement over the 8 series and obviously the 4 series. Bind8 has had quote a few serious 'sploits especially on versions that major *cough* redhat *cough* carried on their 6.2 distro which *a lot* of people run.

I'd rather someone call me paranoid when it comes to security then to get r00ted. I may switch to Bind9 in a few versions, I am certainly not a DJB-zealot but I do like the syntax of setting up records in tinydns compared to Bind's zones.

ymmv,
davidu

Originally posted by DavidU

security then to get r00ted. I may switch to Bind9 in a few versions, I am certainly not a DJB-zealot but I do like the syntax of setting up records in tinydns compared to Bind's zones.

Well, in some ways it has an advantage.

I use my own version of Bind that reads zones directly from a SQL database. I have then built a PHP interface for manipulating every piece of information in DNS. It makes administrering DNS an easy job :-)

The nice thing is, that there is absolutely no text configuration files for bind - and you get rid of that tiring delay (from when you add new information to the point where it is "publically available").

It also makes it possible to log a lot of information. I have query graphs, load graphs, query logs, raw data, etc. Therefore I know exactly which servers are loaded and also which zones are queried the most. I also have statistics over who queries my server a lot...

I use that in hosting projects like ordb.org. I can instantly see which mailservers has started blocking spam.

--
Jens Kristian Søgaard, Mermaid Consulting I/S,
jens@mermaidconsulting.dk,
http://www.mermaidconsulting.com/

Originally posted by jks

I use my own version of Bind that reads zones directly from a SQL database. I have then built a PHP interface for manipulating every piece of information in DNS. It makes administrering DNS an easy job :-)

We've done the same. ;-) We are using a SQL frontend to everydns.net and then having tinydns pull the data from sql.


The nice thing is, that there is absolutely no text configuration files for bind - and you get rid of that tiring delay (from when you add new information to the point where it is "publically available").

We still have a <60 second delay but we haven't found it to be an issue. We have two fixes, one is a live sql database that gets parsed on each lookup or we can just push the data the second we update it with a simple command we wrote.


It also makes it possible to log a lot of information. I have query graphs, load graphs, query logs, raw data, etc. Therefore I know exactly which servers are loaded and also which zones are queried the most. I also have statistics over who queries my server a lot...

Yep, exactly...we post-process out logs every 20 minutes...not live, but for our needs its good enough...if we switched to a fully db backed solution it would be live. (obviously)

I use that in hosting projects like ordb.org. I can instantly see which mailservers has started blocking spam.


Cool, I think I read about ordb.org on the qmail mailing list. Its the ORBS replacement correct?

-davidu

Originally posted by DavidU

We've done the same. ;-) We are using a SQL frontend to everydns.net and then having tinydns pull the data from sql.

What type of database do you use? (MySQL, PostgreSQL, or?)

How fast is your server on queries? (to localhost - in ms)


We still have a <60 second delay but we haven't found it to be an issue. We have two fixes, one is a live sql database that gets parsed on each lookup or we can just push the data the second we update it with a simple command we wrote.

I've written my own caching system (in C).. this makes it possible to serve queries fast and still have a reasonable low delay (it's currently set to 10 sec, but it can be changed simply by changing a field in a database).



Yep, exactly...we post-process out logs every 20 minutes...not live, but for our needs its good enough...if we switched to a fully db backed solution it would be live. (obviously)


Of course. I use a live system recording into a DBMS. I then have a perl daemon running to filter query logs into something useful.


Cool, I think I read about ordb.org on the qmail mailing list. Its the ORBS replacement correct?

Correct. The list is on relays.ordb.org - so setup your mailserver to block spam today! :-)

--
Jens Kristian Søgaard, Mermaid Consulting I/S,
jens@mermaidconsulting.dk,
http://www.mermaidconsulting.com/

So what about if i had 2 servers. One in the UK and one in the USA.
Say i host 100 sites, i have 50 on one sevrer and 50 on another.
How easy would it be so when server 1 went down, server 2 would be able to take over control?
are we talking mega bandwidth here? or impossible?

i want to have near enuff 100% uptime and i see this as a good way since its 2 different locations.

Regards

Craig

Originally posted by Craig
So what about if i had 2 servers. One in the UK and one in the USA.
Say i host 100 sites, i have 50 on one sevrer and 50 on another.
How easy would it be so when server 1 went down, server 2 would be able to take over control?
are we talking mega bandwidth here? or impossible?


Well, you would have 100% uptime on DNS. If one server goes down - the other one responds to all queries.

There is not any bandwidth issue there.

But to me it sounds like you have misunderstood something. DNS is not all! - Your sites will not be functioning if the server is down, but DNS is working. For that you would also need to fully replicate site content - and have double A records - and remember to pull out the A record when the server goes down.

For email it's easier. Here you can just have two MX records for every domain - and then the other server functions as a backup if one goes down.

--
Jens Kristian Søgaard, Mermaid Consulting I/S,
jens@mermaidconsulting.dk,
http://www.mermaidconsulting.com/

Originally posted by jks


Well, you would have 100% uptime on DNS. If one server goes down - the other one responds to all queries.

There is not any bandwidth issue there.

But to me it sounds like you have misunderstood something. DNS is not all! - Your sites will not be functioning if the server is down, but DNS is working. For that you would also need to fully replicate site content - and have double A records - and remember to pull out the A record when the server goes down.

For email it's easier. Here you can just have two MX records for every domain - and then the other server functions as a backup if one goes down.

--
Jens Kristian Søgaard, Mermaid Consulting I/S,
jens@mermaidconsulting.dk,
http://www.mermaidconsulting.com/


There's no way I could have said it better. Totally on the mark Jens.

-davidu

I do not think that i have put my thoughts clearly.
Im looking for 100% uptime, i understand how DNS works but i want something so if for example, UK server dies everything, will be handled by the USA server.
such as sites, email, ftp etc.

Im thinking along the lines of 2 seperate hd's using raid.. so there is a mirror of each site. I know this is possible if the computers are located togeather.. but isthis possible country>country?
or am i asking abit much? :]

Regards

Craig

Originally posted by jks
...
But to me it sounds like you have misunderstood something. DNS is not all! - Your sites will not be functioning if the server is down, but DNS is working. For that you would also need to fully replicate site content - and have double A records - and remember to pull out the A record when the server goes down.

Ok, my query is much the same as Craigs on this one...

From what I understand (and please note I know nothing about the technicalities of DNS) email will essentially head to the up server but HTTP requests will die at the dead primary server.

Now from the rest of the above there's something called an A record but it sounds like you'd physically need to remove the dead servers A record and have a second one for the backup server in place in order for the backup server to start receiving the HTTP request...

Is this a fair interpretation of the above...

Either way can one or other of you please explain what that means in practical terms and as laymanish as you can manage... :blush: I'm here to learn :D

Thanks

Originally posted by WebSnail.net


Now from the rest of the above there's something called an A record but it sounds like you'd physically need to remove the dead servers A record and have a second one for the backup server in place in order for the backup server to start receiving the HTTP request...






Here's what happens:



1. You set up two A records. One pointing to your "primary" server, and one pointing to your backup servers.



2. Now web requests will be split amongst the two servers. Approximately half of the requests will go to the primary, the other half will go to the backup.



Now if one server should crash, one half of the requests will go to a dead server. Not good. So you remove one of the A records from DNS - and all your requests now go to the running server - and everything is fine.



Removing an A record just means editing a textfile and removing a line - or it could be just clicking a button on a webpage (if you use a dns management system like ours). I.e. it's very easy and no trouble at all.

The hard part is getting your website to work with one half of the requests going one place and others to a different server. If you only server static HTML, images, video or similar - it's no problem. But if you have a website that runs off a database that is dynamically updated by the viewers -- then it's hard. You have to make some effort to get the two databases synchronized (replication).

<<MOD EDIT:>>
You may not post URLs in your posts. Please set up a signature in your profile.
<</MOD EDIT>>

Hi Jens,

Ok that sounds about right... I've figured out a theoretical way to do the dynamic stuff so I'm aware of that issue...

Now as far as the primary/secondary issue I read some info here:
http://www.devshed.com/Books/ProApache/page6.html

That seems to indicate there's a way to get the primary to get 100% of the hits unless it's dead in which case the secondary takes over...

Could you let me know if the article is out to lunch, applies to a different situation or if it could work...

If it's feasible then could you go over the article and give an example of how to set up server 1, server 1 and the DNS record for example.com.

Thanks for all the help so far it's interesting albeit complex stuff.

Originally posted by WebSnail.net
Now as far as the primary/secondary issue I read some info here:

http://www.devshed.com/Books/ProApache/page6.html

That seems to indicate there's a way to get the primary to get 100% of the hits unless it's dead in which case the secondary takes over...

Could you let me know if the article is out to lunch, applies to a different situation or if it could work...


Well, the author of the article misunderstood some stuff about DNS - which makes his first proposal not work (the one that 100% of the hits go to one server). He assumes that all clients will ask the primary DNS server first -- that is simply not the case in the real world. Therefore his idea falls to the ground, and it will split hits between the two servers just as simple A-record round-robin DNS would.

I have tested it with our own DNS system, using a slightly different approach to getting the same. We use simple ping to check if the primary server is up - if it's up, it's IP will be given out. If it's down, the IP of the backup server will be given out. This method guarantees that the hits will not be split 50/50 between the two servers.

<<MOD EDIT:>>
You may not post URLs in your posts. Please set up a signature in your profile.
<</MOD EDIT>>

Originally posted by jks
...(snip)

I have tested it with our own DNS system, using a slightly different approach to getting the same. We use simple ping to check if the primary server is up - if it's up, it's IP will be given out. If it's down, the IP of the backup server will be given out. This method guarantees that the hits will not be split 50/50 between the two servers.

I'm assuming this requires some coding direct into the guts of the dns server... would you be willing to share info at all?

Originally posted by WebSnail.net




I'm assuming this requires some coding direct into the guts of the dns server... would you be willing to share info at all?

Yes, it does. We have coded our own DNS management system. It runs with BInd9 directly off a SQL database - with an easy webbased frontend that is made so you easily can manage a lot of domains. We're currently running it with 1500+ domains - no problems.

We originally made it for in-house use, but we have now decided to market it as a product. Our designer is currently making a webpage to market the system.

<<MOD EDIT:>>
You may not post URLs in your posts. Please set up a signature in your profile.
<</MOD EDIT>>

Originally posted by jks


Yes, it does. We have coded our own DNS management system. It runs with BInd9 directly off a SQL database - with an easy webbased frontend that is made so you easily can manage a lot of domains. We're currently running it with 1500+ domains - no problems.

We originally made it for in-house use, but we have now decided to market it as a product. Our designer is currently making a webpage to market the system.

Well I can guarantee it'll be too rich for my blood, although, just as a thought. You might want to consider marketing a small scale DNS service that essentially offers a system for single or small numbers of domains in the way we've been discussing.

Worth considering.. I know it's something I'd be able to pitch to one or two of my customers especially if the price was reasonable.

Thanks for clarifying a few things though.

Originally posted by WebSnail.net
Well I can guarantee it'll be too rich for my blood, although, just as a thought. You might want to consider marketing a small scale DNS service that essentially offers a system for single or small numbers of domains in the way we've been discussing.



Worth considering.. I know it's something I'd be able to pitch to one or two of my customers especially if the price was reasonable.

We were actually thinking of making a pricing scheme based on the number of domains. I.e. so you could buy a 100-domain version, 250, 500, 1500, etc. version. We are aware that most hosting businesses can't afford paying that much.

<<MOD EDIT:>>
You may not post URLs in your posts. Please set up a signature in your profile.
<</MOD EDIT>>

Originally posted by jks


We were actually thinking of making a pricing scheme based on the number of domains. I.e. so you could buy a 100-domain version, 250, 500, 1500, etc. version. We are aware that most hosting businesses can't afford paying that much.

You'll laugh.. in my case I'd be interested in something closer to a 25 - 50 domain version... I kid you not. I'm a SMALL web host with even smaller clients.

If you'd consider a poor small :bawling: web host... *cue: grovelling* ;)

I'd still be interested in the pricing though I must admit.