Domains to process: 566
Available domains written to file: 420
Run time: 1.32230615616
I decided that 25 - 50 results per second was too slow so I incorporated the example given into a threaded version which follows in this post. Depending on the state of your name server's cache, between 5 and 10 workers will probably do the job but regardless, its processing over 400 domains per second on a somewhat old desktop of mine. That ought to be fast enough.
Taking the example code of mine that I linked to earlier, you can see that the core code for checking the domains is only 3 lines of name server checking code plus a few lines here and there to read candidates from a test file and then write them back. Writing the python code to do the actual work took all of a few minutes. Adding in option parsing and threading took a little more time, mostly because I don't write threaded code very often. But still measured in minutes, not hours.
In order to run this code you'll need python installed of course, plus download and install py-dns from http://pydns.sourceforge.net
. Installing most python packages is dirt simple - usually involves no more than:
python setup.py install
Have at it.
(used PHP hilighter just to make this more readable on-line; it is Python of course)
A multi-threaded DNS query solution, originally developed to check a
large list of candidate domain names -- if no NS is present, the candidate
likely does not have a domain registration.
(C) mwatkins, a WebHostingTalk.com user. Use for any purpose you like provided
attribution is provided.
# install PyDNS: http://pydns.sourceforge.net
# FreeBSD: /usr/ports/dns/py-dns
from optparse import OptionParser
WORKERS = 10
def has_ns (domain):
"""(domain:string) -> boolean
query = DNS.Request(domain, qtype="NS").req()
status = query.header['status']
return (domain, status)
"""(path:string) -> [string]
Returns domains from a text file as a list of strings.
return [domain.strip() for domain in open(path).readlines()
def __init__(self, work_queue, result_queue):
self.__queue = work_queue
self.__result = result_queue
domain = self.__queue.get()
if domain is None:
"""(domains:sequence) -> ([string], [string])
Processes a sequence of domain strings and returns two lists
containing those that are potentially available for registration,
and those which have name servers defined and are therefore unavailable.
A final check for availability against a registrar whois API is recommended.
work = Queue.Queue()
result = Queue.Queue()
for domain in domains:
# add work terminators
for i in range(WORKERS):
workers = [Worker(work, result) for i in range(WORKERS)]
for worker in workers:
# wait for workers to complete
for w in workers:
available = 
unavailable = 
while not result.empty():
domain, status = result.get(0)
if status == 'NXDOMAIN':
return available, unavailable
parser = OptionParser()
parser.set_description('Checks domain names for available name servers. '
'Domain names with no name servers defined are potentially available '
'for registration. A final check for availability against a registrar '
'whois API is recommended.')
'-f', '--inputfile', dest='input_file', default='domains.txt',
help='A file containing one domain name per line ')
'-o', '--outputfile', dest='output_file', default='available.txt',
help=('A file to contain the candidate domains which may '
'be available. Note: This file is overwritten on every run.'))
'-s', '--server', dest='dns_server', default=None,
help='Force use of a specific name server, overriding /etc/resolve.conf.')
'-t', '--threads', dest='workers', default=10, type=int,
help='Number of worker threads')
'-c', '--cache', action="store_true", dest='use_cache', default=False,
help=('Use contents of <outputfile>, if it exists, as a cache. '
'In other words, do not requery domains that have been found '
'to be available. Newly found "available" domains will be '
'added to the cache in <outputfile>.'))
options, args = parser.parse_args()
domains = get_domains_from_file(options.input_file)
cached_domains = 
WORKERS = options.workers
DNS.defaults['server'] = [options.dns_server,]
cached_domains = get_domains_from_file(options.output_file)
except IOError, e:
print "INFO: Skipping cache", e
# we don't want to check those that already are in the output file
domains = [domain for domain in domains if domain not in
start = time.time()
print "Domains to process:", len(domains)
avail, unavail = process_domains(domains)
fout = open(options.output_file, 'w')
# we'll add cached_domains back to the output ( if the option wasn't specified)
result = sorted(avail + cached_domains)
fout.writelines(['%s\n' % domain for domain in result])
print "Available domains written to file:", len(result)
print "Run time:", time.time() - start
if __name__ == '__main__':