Overview

Luddite: a person opposed to new technology or ways of working.

In the realm of artificial intelligence, Large Language Models (LLMs) and generative AI stands out as a beacon of progress. Not only are they shaping the way we interact with technology, but they’re also playing a pivotal role in enhancing the content you read, including this blog post.

Before you jump to conclusions, let me assure you this isn’t another hyperbolic piece touting AI as the panacea for all of humanity’s challenges. Rather, our focus is on a more pressing and pervasive issue: the persistent threat of phishing and fraud on the Internet. Despite the Internet’s three-decade history, it remains a fertile ground for cybercriminals, with phishing and fraud representing a multi-billion dollar industry. The Federal Trade Commission (FTC) reported receiving 2.4 million fraud complaints in 2022 alone, amounting to staggering losses of $8.8 billion.

But what does this have to do with Advanced AI? The connection lies in the concerns raised by the cybersecurity community regarding these advancements. Often dismissed as Luddites resistant to change, these experts highlight valid apprehensions about the security implications of new technologies.

A common argument from tech enthusiasts is the necessity to adopt and adapt to new technologies, despite their inherent security flaws. They often point to the early days of the Internet, a digital ‘Wild West,’ believing that, like the Internet, these issues will resolve over time. However, this perspective overlooks a crucial fact: the Internet remains a battleground, rife with security challenges.

This blog post aims to shed light on the harsh reality of the digital landscape. By delving into the depths of daily phishing and fraud incidents, we hope to provide a clear-eyed view of the current state of Internet security, and evidence of how advanced AI is already being abused.

The Digital ‘Wild West’

What if we told you at this point in time, most of the household brands you’re familiar with, and their customers are actively being targeted by an endless barrage of phishing campaigns?

Utility Software Often Targeted

In addition to targeting household brands we’re all familiar with, cyberattackers craft sophisticated phishing sites targeting users of utility and remote desktop software. They deceive users into downloading what appears to be legitimate software, which, unbeknownst to the victim, includes a malicious payload. This malware operates covertly, recording every keystroke and silently transmitting personal and confidential information directly to the attackers.

This stealthy method underscores the importance of vigilant cybersecurity practices to protect against such insidious threats.

Creating Fictitious Banks and Universities

In the cases we’ve highlighted in other blogs, we detailed how fraudsters meticulously impersonate highly reputable brands, aiming their schemes at unsuspecting customers. However, their deceptive tactics have evolved beyond mere impersonation. Our investigations have unearthed a more elaborate ruse: the creation of entirely fictitious entities.

Among these, fake banking websites are common, yet a particularly audacious discovery was a network of counterfeit universities. These fraudsters didn’t stop at creating sham institutions; they went a step further, fabricating websites for non-existent accrediting bodies to lend an air of legitimacy to these phantom universities. This revelation has captivated our readers, making it our most popular blog post to date.

Advertisements for Phishing Delivery

Attackers are utilizing Google Ads for phishing by creating advertisements that lead to malicious sites. These ads often appear at the top of Google search results, masquerading as legitimate services. Users clicking on these ads are redirected to phishing sites where they might unknowingly divulge sensitive information.

This strategy exploits users’ trust in the legitimacy of top search results, making it a particularly effective method of phishing. The technique poses significant challenges for detection and prevention, underscoring the need for increased vigilance from users, tech workers, and platforms like Google. For a detailed explanation, you can read our blog post.

Brian Krebs also recently posted on the topic here.

CloudFlare

Attackers have been known to utilize services like Cloudflare to protect their phishing sites, leveraging its capabilities to enhance the security and resilience of their fraudulent operations. Cloudflare, primarily a content delivery network and security company, provides services that can shield websites from common cyber threats and optimize their performance. In the context of phishing, attackers use Cloudflare to mask the true hosting location of their sites, making it harder for cybersecurity experts and law enforcement to track them down.

Additionally, Cloudflare’s infrastructure can protect these illicit sites from DDoS (Distributed Denial of Service) attacks, ironically safeguarding the phishing operations from disruptions caused by other cyberattackers or take-down attempts. This misuse of legitimate cybersecurity tools highlights the sophisticated methods employed by cybercriminals to ensure the longevity and effectiveness of their phishing campaigns. It also poses a challenge for cybersecurity professionals who must find ways to combat such misuse while maintaining the efficacy of these tools for legitimate users.

In the example below, we see the attackers have crafted a Microsoft phishing site, and are protecting their site using Cloudflare.

Double-edged Nature of Technology Advancement

In conclusion, the evolution of phishing fraud, as detailed in our exploration of its trends, underscores a broader narrative about the double-edged nature of technological advancement. Language Learning Models (LLMs), much like the Internet, embody the pinnacle of human innovation, offering boundless opportunities for enhancing communication, creativity, and problem-solving.

However, as history has consistently shown, every technological leap is accompanied by its exploitation for nefarious purposes. LLMs, despite their vast potential for good, are not immune to this pattern and will inevitably be manipulated by some to perpetrate fraud and deception. This reality does not diminish the value or the necessity of these technologies; instead, it highlights the importance of vigilant, adaptive security measures and ethical frameworks.

We wrote about DeepFakes and concerns for the future of Phishing, back in 2021, and now seems that future has become a reality. There have already been documented instances where individuals were targeted by sophisticated scams involving phone calls from supposed “family members” in distress. In these scams, attackers employ generative AI technologies to create highly convincing audio or video simulations of the victims’ relatives. These artificial reproductions are so realistic that many are deceived into believing they are genuinely interacting with their loved ones, only to later discover that it was a meticulously crafted illusion by fraudsters.

We’re also beginning to see the usage of ‘Dark AI Models’, which are artificial intelligence models built specifically for nefarious purposes. The AI tools we’re most familiar with, such as ChatGPT, have safeguards in place preventing users from asking the model to generate content that is malicious in nature. Cyber-criminals can build and train their own models, and eliminate safeguards allowing their users to do things such as generate fake IDs. 404 Media recently featured an article describing an underground site ‘OnlyFake’ used to generate fake IDs. 404 Media Post

Protect Yourself

There’s various steps you can take to protect yourself, your organization, or your customers. All of which we would consider to fall under your ‘Domain’.

For End Users

For everyone who is a general user, here are some tips to better protect yourself.

• Exercise caution with advertisements
  • A persistent threat for over a decade, malicious advertisements demand your vigilance. Always approach ads with caution, skeptical of where they might lead, even if they claim to represent a genuine entity.
  • Don’t autopilot click the first result on Google without thinking about it.
• For suspicious sites
  • Domain Legitimacy Tools: Before trusting a website, verify its domain’s authenticity. A tell-tale sign of a scam is a recently registered domain claiming to represent a well-established brand. Tools like ICANN’s Domain Lookup can be invaluable.
    • ICANN’s Domain Lookup Page
  • Safe Browsing Utilities: Use services like URLScan to safely preview the contents of a website without risking your security. This approach keeps your computer safe while allowing you to inspect potentially dubious sites from a distance.
    • URLScan
• Be wary of security features
  • HTTPS Misconceptions: While HTTPS indicates encrypted traffic, it doesn’t guarantee the site’s legitimacy. Scammers often exploit this feature to create a false sense of security.
  • Cloudflare Caution: Scammers utilize services like Cloudflare just as legitimate sites do. A site’s use of Cloudflare should not be taken as a sure sign of its authenticity.
• Define safe words for your close family.
  • Safe words can serve as a crucial security measure in scenarios involving AI-generated impersonations of family members or acquaintances. By establishing a pre-agreed, unique word or phrase that is known only to close contacts, individuals can create a simple yet effective verification mechanism. In the event of a suspicious call or message claiming to be from a family member or friend in distress, the recipient can ask for the safe word.
  • If the caller is unable to provide the correct safe word, this serves as a clear indication that the communication may not be genuine and is potentially a sophisticated scam utilizing generative AI.

For Cyber-security Employees or Leadership

For professionals tasked with cybersecurity responsibilities within their organizations, aiming to safeguard their brands, customers, and employees from cyber threats, here are some tips.

• Consider Monitoring: Determine if your organization would benefit from continuous monitoring for these types of threats.
  • If your organization is budget restrained, we’ve prepared a Free and Open-Source guide to help you to get started with basic coverage.
  • FOSS Domain Monitoring Guide