Difficulty Level: 2/5
If you have fallen to the dark side of SEO and decide its time to build or sell some links… Then having a domain with High PR sites pointing live Do-Follow links at it is extremely valuable. Basically what we are going to do is crawl a large website like huffpost, buzzfeed, cnn and collect as many pages of theirs as we can. Once we have a LARGE list of urls from these sites, we can find all the Do-follow links to external websites. After you have the list, you can bulk check those domains so you can find the free to buy domains.
If you do not own a copy of scrapebox, you can find a discounted link here for $67 – this discount code is from Blackhatworld ( not mine ) Its normally $99.95 without it.
Step 1: Crawling A High PR Domain
Generally I find it easier to “Grab links by crawling a site” at first, to get 50,000-250,000+ links to start with. Then Running the Link extractor add on program. You will find this feature under the Grab / Check.
Enter in the domain in the format of https://domain.com or http://domain.com and Start the program. When the crawl is finished, import the list of urls into the harvester.
You will find the list of urls in the scrapebox root folder – under Grabbed Links. Right click on the URLs Harvested box and Import and replace current list with this.
Remove duplicate URLs
Step 2: Extracting Links from websites
Head over to the addons top menu – Install and run Link Extractor.
Link Extractor Configuration
To make sure you have not missed any internal links ( I have here ), Run another Internal Link scrape with Link Extractor. Make sure Sub Domains are treated as internal links.
Finding Do Follow Links to External Domains
After you have a larger list of internal links from the main site you are crawling, you can start crawling these for Do-follow external links. The nice thing i like about this add on is the filters. If you are crawling a website with ads, you are going to get a super messy list of junk. You can also use it to find partial/exact match domains, it will just take a very large list of internal URLs to find free domains that match those filters.
Cleaning up the List of Data
Import the list of external Do follow links that you have just extracted.
Trim the list to its Root domain.
and Remove/Filter – Duplicate URLs.
Step 3: Check for Unregistered Domains
Under Grab/Check – You will find Check Unregistered Domains.
Start the domain lookup, and wait for this to finish. If you are running a large report, you might have to tweak the settings abit. ( You will likely need to run WHOIS, i didn’t find any domains on this pass without it )
Export the list of available domains, and view the txt file.
IOM.EDU is a dead domain, but we sadly cant buy edu domains. Everything else looks like its free to buy right now, and should have a live link from huffpost pointing at it.
Manually Reviewing The Data
If we go to google site search, we can verify the link is there, and what page it is on.
Enter your websites URL: The domain you crawled.
Enter a Search Phrase: Root domain/Name of site you are wanting to buy
Here is where that link is located at – and yes it is a free to buy domain. There are “No Set rules” On how to use scrapebox, you just gotta force it to do what you want.