Background
Understanding that not all businesses or individuals can afford a dedicated Domain Monitoring service—yet still deserve robust digital protection—we have decided to curate a comprehensive guide on free and open-source tools one can use to monitor their Domain. This initiative underscores our commitment to the wider community, emphasizing that while our premium services offer unparalleled protection, we believe in equipping everyone with the knowledge to safeguard their domain, irrespective of budget constraints.
What Is Your Domain?
At DomainGuard, we perceive a domain not merely as a web address but as an encompassing representation of an organization’s identity — its name, brand, employees and all associated technical infrastructure are vulnerable to potential threats or impersonation. Our signature ‘Domain Threat Assessment’ examines and evaluates a client’s Domain based on six distinct criteria, offering invaluable insights into potential vulnerabilities.
We present a handpicked selection of Free and Open Source Software (FOSS) tools, empowering individuals and businesses to better Guard their Domain.
Approved Assets
We begin by cataloging authorized assets within a specified domain. This includes official domain names like the primary company website, social media profiles, employee accounts, and various systems accessible online. In this article, we’ll specifically delve into approved internet domains and their subdomains.
The goal for this stage is to have an organized inventory of approved domains and subdomains.
Tools to Identify Approved Domains
If you aim to monitor your own organization, pinpointing your authorized domains is straightforward, especially if you maintain an internal inventory of them. If you haven’t compiled such a list, it’s a prudent step to consider.
For the sake of this illustration, let’s assume we are conducting reconnaissance on an entity with which we are not familiar, aiming to identify their approved domains.
Identify Main Domain
The most obvious place to start is by performing a search engine query for your company and their affiliated brands. Additionally, a quick Google dork such as site:example.com
will produce some good results. However, if you want to omit the main company website domain and list subdomains, add a hyphen ‘-’ such as site:example.com -www.example.com
. More subdomain enumeration is covered below.
Another good way to identify your primary domains and brand domains is by working with your internal teams that manage your domain registration such as Godaddy, Amazon, NameCheap, etc. They can easily pull a list from these entities, and you will be on your way to the next phases.
Identify Related Domains by Web Tools
There are several websites that contain domain databases and allow you to search by keyword, or related registration information such as email address. With the increase of privacy protection on Domains, the results can be less accurate but are still worth checking. All the websites below provide a simple interface to allow you to query domains by keyword, and attempt to identify other related domains.
Identify Related Domains by Spidering
After discovering the authorized domain(s), it’s important to explore associated domains. A straightforward method to discover associated domains, is by seeing domains are linked from the primary website. By spidering the main website, one can determine if it connects to any other domains or subdomains.
We can use the tool, GoSpider, to help us spider in order to identify related domains from the main domain.
Command to run spider against a target domain, and filter out any results that are the target domain.
docker run gospider -s "https://example.com" -c 10 -d 1 | grep -v "example.com"
Output
[href] - https://www.exampleobjects.com/en_US/i/icon/pp_favicon_x.ico
[href] - https://www.exampleobjects.com/en_US/i/pui/apple-touch-icon.png
[href] - https://www.exampleobjects.com/web/res/4d6/dsa13/css/app.css
In this example, our target domain was example.com
and we identified another domain referenced from the main website, exampleobjects.com
.
Identify Subdomains
After pinpointing your main domains, it’s essential to compile a list of subdomains to ensure a comprehensive domain inventory. Various methods exist for enumerating subdomains, and we’ll highlight a few of these approaches below.
GoSpider Subdomains
We can use our GoSpider method again, this time focusing on subdomains as opposed to potentially related sites identified by spidering the main website. Imagine for example if the site example.com
contains a link to their application app.example.com
.
Command to run GoSpider and filter only for subdomains.
docker run gospider -s "https://example.com" -c 10 -d 1 | grep subdomain
Output
[subdomains] - https://khms0.example.com
[subdomains] - https://khms1.example.com
[subdomains] - https://maps.example.com
[subdomains] - https://www.example.com
[subdomains] - https://developers.example.com
Amass Subdomains
Amass is another popular tool used to identify subdomains.
Command to run Amass for domain enumeration.
docker run amass enum -d example.com
Sample output grepped for only subdomains
cat amass-output | grep .example.com
videos.example.com (FQDN)
reportincident.example.com (FQDN)
remote.example.com (FQDN)
forest.example.com (FQDN)
cmzxks-test-coip.example.com (FQDN)
crt.sh Subdomain
crt.sh is an online tool dedicated to logging certificates. It’s especially handy for spotting subdomains of a specified domain, provided a certificate record was established for that subdomain. It offers a user-friendly alternative for those hesitant to use command line interfaces.
Other Web Tools for Subdomain Enumeration
Below are links to other web based tools that can be useful for Subdomain enumeration.
Easily View Domains and Subdomains
At this point we should have achieved our goal of having an organized list of approved domains and subdomains. Great, now what?
Your next step is to periodically review this list, keeping an eye out for unfamiliar entries which could indicate new assets for the organization. While numerous tools and monitoring methods exist for overseeing your approved domain list, the most time-effective and direct approach is to connect to these domains via a browser to observe their content.
This can be done manually, or you can use a CLI tool, gowitness , to automate this process.
GoWitness allows you to pass in your list of domains and subdomains, and it will attempt to screenshot each of the URLs and has a nice web interface to allow you to quickly view your list of approved domains.
Docker command to run gowitness and pass in your list of domains\subdomains from the current directory, where GoWitness will save data.
docker run --rm -v $(pwd):/data leonjza/gowitness gowitness file -f domains
Docker command to view output from gowitness web interface referencing data in your current directory.
docker run --rm -v $(pwd):/data -p7171:7171 leonjza/gowitness gowitness server --address :7171
Identify Lookalike Domains
In addition to identifying Approved Domains and Subdomains to keep an eye on, you’ll want to identify lookalike domains as these domains could be used for phishing or brand impersonation.
Google Alerts for Content Keywords
A very easy thing anyone can do is to set up Google Alerts notifications for keywords related to their domain or brand. This will only identify websites hosting content related to your keywords, so it may not be the fastest or most accurate, but is a good free option to consider.
dnstwist for identifying Lookalike Domains
Another tool to add to your arsenal for identifying lookalike domains is dnstwist which has a web interface as well as a github for command line access. dnstwist works by generating permutations of a domain name you provide, and then attempting to resolve those permutations to see if the domains are active.
Phishing Catcher for identifying Lookalike Domains
Phishing Catcher is another tool you can utilize, which runs on a continuous basis to identify lookalike domains. Phishing Catcher works by analyzing TLS certificate issuances reported to Certificate Transparency logs. By creating keywords relevant to your Domain, you can have a continuous stream of lookalike domains as they requst certificates.
In the example below, which is a reference to an image hosted on the Github page for Phishing Catcher, you can see they are using the tool to identify lookalike domains
related to paypal
.
Easily View Lookalike Domains
Just as our goal with the Approved Domains section was to have an organized list of approved domains and subdomains, our goal with this section is to have an organized list of lookalike domains. From here, we can use the web screenshotting tool mentioned earlier, gowitness, to once again screenshot and examine lookalike domains.
Subdomain Takeover
The last area we’ll focus on this blog post, is Subdomain Takeover. Subdomain Takeover’s have become increasingly popular as organizations are using software providers in the cloud. As DNS references are made to certain services, if those references are not kept up to date, there could be a scenario whereby someone could claim ownership of your Subdomain with an existing service.
For example, if our organization wants to host a company blog at blog.example.com
, and this blog contains an A or CNAME record to a hosting provider. We’ll make up a hosting provider for this blog post, “Hostlify”. Let’s imagine this blog was created as a test
to prove Hostlify would be a suitable business use case for this company to host their blog. Some time has passed, and the individual responsible for setting up the initial implementation decided they no longer wanted to use Hostlify, so they deleted their account
with Hostlify, but never went back and cleaned up the DNS records for blog.example.com
.
Now imagine an attacker goes into Hostlify, and says “Hey, I’m all configured to publish my website, here’s my DNS record: blog.example.com
” and this DNS record has already been validated and continues to point to Hostlify. The attacker
at this point can publish any web content, under blog.example.com
and has complete control of the website the DNS records are still pointing to.
Fortunately, the solution is straightforward: tidy up your DNS records.
Using your list of approved subdomains and screenshots for these subdomains, analyze the subdomains to determine if a takeover scenario is possible.
These links may be of assistance:
Dangling IP Takeover
It’s also pertinent to touch on ‘Dangling IP Takeover,’ which closely mirrors the Subdomain takeover scenario described earlier. To illustrate, suppose you have a DNS record directing to an IP address with a cloud provider, but that IP address pertains to a resource you’ve either relinquished or decommissioned. Should a new user on the cloud platform be assigned that same IP address and your DNS still points to it, they might intercept traffic, data, or requests intended for your original resource. Such a situation paves the way for potential data breaches, man-in-the-middle attacks, phishing, and other security threats.
Fortunately, the solution is straightforward: tidy up your DNS records.
Summary
Regular domain, subdomain, and lookalike domain monitoring can prevent a myriad of potential threats. Thankfully, with the free and open-source tools available, securing one’s domain has never been more accessible.
While the array of free and open-source tools provides invaluable resources for domain monitoring, those seeking a more streamlined, comprehensive, and hassle-free solution might opt for a commercial service. When considering a commercial domain monitoring service, it’s crucial to select one that offers:
- Continuous monitoring to capture real-time threats.
- Change tracking to highlight when web content changes from benign to malicious.
- Certificate monitoring to ensure that all modifications and certificate issuances are diligently overseen.
- Weekly reporting to provide consistent updates on domain health.
- Early warning alert notifications on new or related domains, to keep users promptly informed of potential impersonations or threats.
- Takedown workflow to facilitate swift resolution.
- A centralized interface to seamlessly track and manage all these features, offering users a holistic view of their Domain.
Need more assistance?
If you found the information above difficult to consume or need additional assistance, please reach us by email at [email protected] or by filling out the contact form below.