Background

Understanding that not all businesses or individuals can afford a dedicated Domain Monitoring service—yet still deserve robust digital protection—we have decided to curate a comprehensive guide on free and open-source tools one can use to monitor their Domain. This initiative underscores our commitment to the wider community, emphasizing that while our premium services offer unparalleled protection, we believe in equipping everyone with the knowledge to safeguard their domain, irrespective of budget constraints.

What Is Your Domain?

At DomainGuard, we perceive a domain not merely as a web address but as an encompassing representation of an organization’s identity — its name, brand, employees and all associated technical infrastructure are vulnerable to potential threats or impersonation. Our signature ‘Domain Threat Assessment’ examines and evaluates a client’s Domain based on six distinct criteria, offering invaluable insights into potential vulnerabilities.

We present a handpicked selection of Free and Open Source Software (FOSS) tools, empowering individuals and businesses to better Guard their Domain.

Approved Assets

We begin by cataloging authorized assets within a specified domain. This includes official domain names like the primary company website, social media profiles, employee accounts, and various systems accessible online. In this article, we’ll specifically delve into approved internet domains and their subdomains.

The goal for this stage is to have an organized inventory of approved domains and subdomains.

Tools to Identify Approved Domains

If you aim to monitor your own organization, pinpointing your authorized domains is straightforward, especially if you maintain an internal inventory of them. If you haven’t compiled such a list, it’s a prudent step to consider.

For the sake of this illustration, let’s assume we are conducting reconnaissance on an entity with which we are not familiar, aiming to identify their approved domains.

Identify Main Domain

The most obvious place to start is by performing a search engine query for your company and their affiliated brands. Additionally, a quick Google dork such as site:example.com will produce some good results. However, if you want to omit the main company website domain and list subdomains, add a hyphen ‘-’ such as site:example.com -www.example.com. More subdomain enumeration is covered below.

Another good way to identify your primary domains and brand domains is by working with your internal teams that manage your domain registration such as Godaddy, Amazon, NameCheap, etc. They can easily pull a list from these entities, and you will be on your way to the next phases.

Identify Related Domains by Web Tools

There are several websites that contain domain databases and allow you to search by keyword, or related registration information such as email address. With the increase of privacy protection on Domains, the results can be less accurate but are still worth checking. All the websites below provide a simple interface to allow you to query domains by keyword, and attempt to identify other related domains.

• ViewDNS
• WHOXY
• Reverse Whois

Identify Related Domains by Spidering

After discovering the authorized domain(s), it’s important to explore associated domains. A straightforward method to discover associated domains, is by seeing domains are linked from the primary website. By spidering the main website, one can determine if it connects to any other domains or subdomains.

We can use the tool, GoSpider, to help us spider in order to identify related domains from the main domain.

Command to run spider against a target domain, and filter out any results that are the target domain.

docker run gospider -s "https://example.com" -c 10 -d 1 | grep -v "example.com"

Output

[href] - https://www.exampleobjects.com/en_US/i/icon/pp_favicon_x.ico
[href] - https://www.exampleobjects.com/en_US/i/pui/apple-touch-icon.png
[href] - https://www.exampleobjects.com/web/res/4d6/dsa13/css/app.css

In this example, our target domain was example.com and we identified another domain referenced from the main website, exampleobjects.com.

Identify Subdomains

After pinpointing your main domains, it’s essential to compile a list of subdomains to ensure a comprehensive domain inventory. Various methods exist for enumerating subdomains, and we’ll highlight a few of these approaches below.

GoSpider Subdomains

We can use our GoSpider method again, this time focusing on subdomains as opposed to potentially related sites identified by spidering the main website. Imagine for example if the site example.com contains a link to their application app.example.com.

Command to run GoSpider and filter only for subdomains.

docker run gospider -s "https://example.com" -c 10 -d 1 | grep subdomain

Output

[subdomains] - https://khms0.example.com
[subdomains] - https://khms1.example.com
[subdomains] - https://maps.example.com
[subdomains] - https://www.example.com 
[subdomains] - https://developers.example.com

Amass Subdomains

Amass is another popular tool used to identify subdomains.

Command to run Amass for domain enumeration.

docker run amass enum -d example.com

Sample output grepped for only subdomains

cat amass-output | grep .example.com

videos.example.com (FQDN)
reportincident.example.com (FQDN)
remote.example.com (FQDN)
forest.example.com (FQDN)
cmzxks-test-coip.example.com (FQDN)

crt.sh Subdomain

crt.sh is an online tool dedicated to logging certificates. It’s especially handy for spotting subdomains of a specified domain, provided a certificate record was established for that subdomain. It offers a user-friendly alternative for those hesitant to use command line interfaces.

Other Web Tools for Subdomain Enumeration

Below are links to other web based tools that can be useful for Subdomain enumeration.

• BinaryEdge
• SecurityTrails
• RiskIQ

Easily View Domains and Subdomains

At this point we should have achieved our goal of having an organized list of approved domains and subdomains. Great, now what?

Your next step is to periodically review this list, keeping an eye out for unfamiliar entries which could indicate new assets for the organization. While numerous tools and monitoring methods exist for overseeing your approved domain list, the most time-effective and direct approach is to connect to these domains via a browser to observe their content.

This can be done manually, or you can use a CLI tool, gowitness , to automate this process.

GoWitness allows you to pass in your list of domains and subdomains, and it will attempt to screenshot each of the URLs and has a nice web interface to allow you to quickly view your list of approved domains.

Docker command to run gowitness and pass in your list of domains\subdomains from the current directory, where GoWitness will save data.

docker run --rm -v $(pwd):/data leonjza/gowitness gowitness file -f domains

Docker command to view output from gowitness web interface referencing data in your current directory.

docker run --rm -v $(pwd):/data -p7171:7171 leonjza/gowitness gowitness server --address :7171

Identify Lookalike Domains

In addition to identifying Approved Domains and Subdomains to keep an eye on, you’ll want to identify lookalike domains as these domains could be used for phishing or brand impersonation.

Google Alerts for Content Keywords

A very easy thing anyone can do is to set up Google Alerts notifications for keywords related to their domain or brand. This will only identify websites hosting content related to your keywords, so it may not be the fastest or most accurate, but is a good free option to consider.

dnstwist for identifying Lookalike Domains

Another tool to add to your arsenal for identifying lookalike domains is dnstwist which has a web interface as well as a github for command line access. dnstwist works by generating permutations of a domain name you provide, and then attempting to resolve those permutations to see if the domains are active.

Phishing Catcher for identifying Lookalike Domains

Phishing Catcher is another tool you can utilize, which runs on a continuous basis to identify lookalike domains. Phishing Catcher works by analyzing TLS certificate issuances reported to Certificate Transparency logs. By creating keywords relevant to your Domain, you can have a continuous stream of lookalike domains as they requst certificates.

In the example below, which is a reference to an image hosted on the Github page for Phishing Catcher, you can see they are using the tool to identify lookalike domains related to paypal.

Easily View Lookalike Domains

Just as our goal with the Approved Domains section was to have an organized list of approved domains and subdomains, our goal with this section is to have an organized list of lookalike domains. From here, we can use the web screenshotting tool mentioned earlier, gowitness, to once again screenshot and examine lookalike domains.

Subdomain Takeover

The last area we’ll focus on this blog post, is Subdomain Takeover. Subdomain Takeover’s have become increasingly popular as organizations are using software providers in the cloud. As DNS references are made to certain services, if those references are not kept up to date, there could be a scenario whereby someone could claim ownership of your Subdomain with an existing service.

For example, if our organization wants to host a company blog at blog.example.com, and this blog contains an A or CNAME record to a hosting provider. We’ll make up a hosting provider for this blog post, “Hostlify”. Let’s imagine this blog was created as a test to prove Hostlify would be a suitable business use case for this company to host their blog. Some time has passed, and the individual responsible for setting up the initial implementation decided they no longer wanted to use Hostlify, so they deleted their account with Hostlify, but never went back and cleaned up the DNS records for blog.example.com.

Now imagine an attacker goes into Hostlify, and says “Hey, I’m all configured to publish my website, here’s my DNS record: blog.example.com” and this DNS record has already been validated and continues to point to Hostlify. The attacker at this point can publish any web content, under blog.example.com and has complete control of the website the DNS records are still pointing to.

Fortunately, the solution is straightforward: tidy up your DNS records.

Using your list of approved subdomains and screenshots for these subdomains, analyze the subdomains to determine if a takeover scenario is possible.

These links may be of assistance:

• Can I takeover XYZ
• subjack

Dangling IP Takeover

It’s also pertinent to touch on ‘Dangling IP Takeover,’ which closely mirrors the Subdomain takeover scenario described earlier. To illustrate, suppose you have a DNS record directing to an IP address with a cloud provider, but that IP address pertains to a resource you’ve either relinquished or decommissioned. Should a new user on the cloud platform be assigned that same IP address and your DNS still points to it, they might intercept traffic, data, or requests intended for your original resource. Such a situation paves the way for potential data breaches, man-in-the-middle attacks, phishing, and other security threats.

Fortunately, the solution is straightforward: tidy up your DNS records.

Summary

Regular domain, subdomain, and lookalike domain monitoring can prevent a myriad of potential threats. Thankfully, with the free and open-source tools available, securing one’s domain has never been more accessible.

While the array of free and open-source tools provides invaluable resources for domain monitoring, those seeking a more streamlined, comprehensive, and hassle-free solution might opt for a commercial service. When considering a commercial domain monitoring service, it’s crucial to select one that offers:

• Continuous monitoring to capture real-time threats.
• Change tracking to highlight when web content changes from benign to malicious.
• Certificate monitoring to ensure that all modifications and certificate issuances are diligently overseen.
• Weekly reporting to provide consistent updates on domain health.
• Early warning alert notifications on new or related domains, to keep users promptly informed of potential impersonations or threats.
• Takedown workflow to facilitate swift resolution.
• A centralized interface to seamlessly track and manage all these features, offering users a holistic view of their Domain.