Domain Investing & Website Development

Domaining site recommended by Domaining.com

The Power of WHOIS

Michael Sumner • Feb 5, 2009 • 19 Comments

Every domain name investor uses WHOIS on a daily basis to check registrant information on domain names, but few understand the mechanics behind it and what you can do with that knowledge.  Consider this your education.  I am not only going to explain how WHOIS data is retrieved, but also give you examples of how you can harness this information to streamline common tasks related to domain investing.  I will be providing working code for most of the examples, but don’t let that frighten you off.  It can be directly uploaded to your server and works out of the box (with the proper server setup).

To understand how WHOIS works, you must first understand that there are two types of WHOIS models, “thick” and “thin”.  In the thick model, registrant information from all registrars is stored centrally, meaning in one location.  This model provides for consistent data in terms of both formatting and the fields presented, and is used by .org and a few ccTLDs.  In the thin model, registrant information is stored by the registrar the domain is managed by.  This model is used by most gTLDs such as .com, .net, .info, and by just about every ccTLD.  At this point you should be wondering: if the data isn’t stored in one location, how do you go about accessing it?

To access registrant information that is stored at the registrar level, you must first determine which registrar the domain is at and what the address of its WHOIS server is.  That is where CenterGate Research Group comes in.  They provide a free service at whois-servers.net that, when queried properly through software, will tell you the registrar, WHOIS server, name servers, creation/expiration date, and status.  The service listens on port 43 at ext.whois-servers.net where ext is the extension of the domain you are checking.  CRG does not limit or throttle the number of requests you can make, so keep that in mind for later.  A response for a thick-model query contains the registrant information.  A response for a thin-model query looks like this:

Domain Name: DNMEDIA.COM
Registrar: GODADDY.COM, INC.
Whois Server: whois.godaddy.com
Referral URL: http://registrar.godaddy.com
Name Server: NS1385.HOSTGATOR.COM
Name Server: NS1386.HOSTGATOR.COM
Status: clientDeleteProhibited
Status: clientRenewProhibited
Status: clientTransferProhibited
Status: clientUpdateProhibited
Updated Date: 23-jan-2009
Creation Date: 11-sep-2005
Expiration Date: 11-sep-2011

The line we care about is “Whois Server”.  We now know where we have to query to get the registrant information.  One more request and we’ve got our data.  Let’s stop for a moment and see what a response looks like from a thin query when a domain is not taken by someone else and is available to register (the response for thick query is “NOT FOUND”):

No match for "TSATWERASFSADF.com".

You’re probably wondering where I am going with this tangent.  One common question I see on the forums is how to do a bulk availability check.  When I say bulk, I’m not talking about 100 or 500, I’m talking about 10,000 or 50,000.  The script below presents you with a text area to paste domains, loops through querying whois-servers.net until all have been checked, and then displays the available ones.  This has two major advantages over checking availability at a registrar.  The first is obviously that you are not limited to 500 results.  The second is that you don’t have to worry about front-running, because the script is executing on your own server and you have complete control.  Nobody will be selling your searches, and thus I recommend using this script to check availability even if it isn’t in bulk.  If you’re scanning a big list, you need to make sure the max execution time for scripts on your server is set to 0 (zero) so it doesn’t time out.

http://www.dnmedia.com/bulk_availability.php

That’s pretty useful, but the real power comes when we do the second query, the one to the actual WHOIS server holding the data.

Let’s say you have a domain name that you’re trying to sell to an end user… you’re probably going to end up sending 50-100 emails.  Having to check the WHOIS for each end user’s domain is really time consuming.  If your WHOIS source displays the email in plain text for you to copy/paste, they usually have a CAPTCHA to slow you down.  Otherwise, like DomainTools, they will usually display the email in an image and you can’t copy it.  Regardless, doing all of those WHOIS queries one at a time just plain sucks.  Well, with the following script you can paste the list of domains you want to email a pitch to, and it will output a list of their email addresses in CSV Format.

http://dnmedia.com/bulk_whois.php

TIP: Enter keywords for the domain you are trying to sell at DomainTools.com Domain Search and it will give you a list of domains that are related to your keywords.   You can use this to find prospects in addition to standard methods such as checking who is bidding in Adwords, who shows up in search engine results, etc.

IMPORTANT NOTICE: It is against the TOS of all registries and registrars to query the WHOIS database for mass, indescriminate, emailings.  Only use the bulk WHOIS script once you have narrowed down your prospect list, and send out a personalized email to each company.  Do not abuse this script… and remember, I’m not responsible for what you do with it.  Spamming can get you slammed with some very heavy fines, so use your best judgement.

I hope you enjoyed this article, and if you have any questions please don’t hesitate to ask.

19 Responses to The Power of WHOIS

  1. Frank Michlick says:

    Great post Michael and great tools. Just to add to this: Registrars do (read “should”) not use the whois to determine availability of domain names, as most registries do provide an alternative realtime availability lookup via their APIs. A lot of domain suggestion tools also us a combination of lookups and zonefiles in order to determine availability of a large number of domain variations in a short time.

  2. Michael Sumner says:

    Frank, thanks for the comment. Whois-Servers.net gets their data directly from the registries, and that is what the script I provided uses to do the availability checks. We have a C# .net version of the program that is multi-threaded and can scan upwards of 25,000 domains a minute, but we’re not really keen on giving that out.

    Zone files are a good method if you’re only scanning gTLDs, but ccTLD zone files are difficult to get your hands on. Plus, the last time I downloaded the .com zone file it was around 1.5 GB, so if you have a lot of zone files to download it can require a massive amount of bandwidth to keep them up to date.

  3. Cathy says:

    Hi
    Great post – I am not a php gal. In the script bulk_availability.php were do I place my list? I have about 100,000 I would like to check for availability. Thanks!

  4. Michael Sumner says:

    @ Cathy

    Sorry, I should have elaborated more, I got a few emails about this as well. What you do is copy the script into a text editor, save it as bulk_availability.php, and then upload it to your server. Then you go to http://www.yoursite.com/path/to/bulk_availability.php in a web browser, replacing yoursite.com with your domain name, and path/to/ with the path you uploaded the script to. It will display a text area for you to paste the list. Then you just click submit and wait. 100,000 domains will take a while to scan though, and if your server isn’t configured correctly you may run into issues with the script timing out. Let me know if you have any problems.

  5. Tim_Myth says:

    WOW! You must be psychic! I was just talking to a friend of mine last night on how to do bulk lookups for our domain http://www.droppertunity.com. We really didn’t have much of an idea on where to start, but thanks to your scripts I have a great understanding of where to go now. Awesome! Thank you! :)

  6. Michael Sumner says:

    My pleasure. We’re actually going to be launching a new drop site soon, so stay tuned for that.

  7. […] I showed you how to create a bulk availability checker using the power of WHOIS.  Today I’d like to follow that up with another interesting use of your new found WHOIS […]

  8. Prahmajankaya says:

    Super-Duper site! I am loving it!! Will come back again – taking you feeds also, Thanks.

  9. Jeff says:

    Very interesting – thanks!

  10. Iceteh says:

    I was very excited at the beginning because I have been looking for something like this for a long time. I entered couple of domain and it works like a charm but then I enter yahoo.com, it returns false =(

  11. Kurt says:

    Hi Michael, just came across your site and PHP script (drop catcher). Useful and very generous of you to give this script out.

    I know you posted this a while back, but just a quick question if you have time.

    Incidentally, you mention above in the Whois post, about ensuring the max execution time for scripts on server is set to 0 (zero). is this something that the user can control within their hosting account at HostGator? Or do you need to ask Hostgator to do at their site?

    Thanks

  12. Bando says:

    Thanks for the code. How would I go about getting the creation and expiration date for a domain instead of the email address? My PHP skills are very limited so I would appreciate your help. Thanks!

  13. Gideon says:

    Could you pretty please tell me how to do what you did in Asp.net, or atleast tell me what your php code is doing.

  14. Chris says:

    This worked awesome!!!… but how can we modify the script to also display the name and phone of the registrant???

  15. chris says:

    What is the max domians I should lookup at one time? is there a cap with the PHP script you gave above?

  16. I am trying to use the bulk_whois.php and it works on my hosting account, but when I try to access my domain (the register is GoDaddy) devanhcrow.com I get a FAILED return.

    Do I have to have Port 43 open? Also, how can I make sure Port 43 is open if it is needed?

  17. Jared says:

    Does this bulk_availability method still work for large numbers of domain names?

    I tried using it and have probably checked less than 1000 domains, and I’m now getting a message about exceeding my whois limit.

  18. Jared says:

    I looked into this further and it appears I am only getting the limit error on the “org” domains. The limit is 4 ORG Domain lookups per minute. Is there another way to check ORG domain availability for hundreds of domains per minute?

  19. Alfred says:

    you need to update the script because the response strings have changed !

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>