AI Scams – AIIP Series on Artificial Intelligence, Part 4
By Arthur Weiss
Editor’s note: This is part of a series covering what’s new and, more importantly, how infopreneurs can maximize the benefits of AI tools.
When I started this blog, my aim was twofold:
- To explain what AI is, how to use it, and how to get the best results from it – that’s the Artificial Intelligence part of the title.
- To combat misinformation and foster critical thinking, so that, as infopreneurs, we can guard against AI-generated hallucinations while also exposing misinformation and disinformation. That’s the Real Information part.
I had intended to continue exploring topics such as effectively prompting AI tools and what to avoid when doing so, making comparisons between AI tools, and offering reviews of some lesser-known tools. I will look at these matters in future posts. Instead, in this post, I will examine how AI can be utilised to analyse scams and support the dissemination of accurate information. This change was prompted by an email I received:
It appeared to be from “Nestle UK Ltd. Supply Chain Management” (robertlawson@nestlesprojects.com) asking whether I could supply a particular type of industrial pump. At first glance, the email looked odd but plausible. It included a UK phone number that linked to the town associated with the address, and the address was for Nestlé. The wording and grammar did not scream out “spam”. It even addressed my initial concern that the request was poorly targeted – perhaps a case of mistaken identity? “The product we request may fall outside your scope of work; hence, we would like you to source the product and supply it to us.”
Curious, I called the number, expecting no answer. But somebody picked up. He acknowledged that the request may have been misdirected, but asked if I could still supply the product. However, the caller’s accent suggested he was not from the UK, making the name “Robert Lawson” immediately suspect.
I then noticed the sender’s domain, “nestlesprojects.com” (with an extra “s”), and knew the mail was a scam. A WHOIS search revealed that the domain had been registered only four days earlier in Nigeria!
I wondered: would AI tools also recognise this as a scam? They did! All the tools I used came through with flying colours, even picking up signs I’d missed. They spotted that nestlesprojects was bogus, stating that official emails from Nestlé would end with @Nestle.com, @uk.nestle.com or something similar, noting that “The domain in your email is not associated with Nestlé and is a common tactic used by scammers to appear legitimate.”
There were two PDF attachments in the email; one provided a different company registration number than the one mentioned in the email. All the AI tools flagged this mismatch, whereas I’d missed it, and it’s a sure sign of fraud. The AI tools also noticed other things: commenting that “large corporations like Nestlé have strict procurement processes,” whereas the email I received suggested a more ad hoc process.
AI tools, famously, often lie (a technical term here: “hallucinate”) and provide fabricated sources. Christopher Kachouroff, a lawyer, was recently lambasted by U.S. District Judge Nina Wang after submitting a court filing citing 30 cases that didn’t exist – and had been fabricated by the AI tool used by Kachouroff or his aide. (See https://lawandcrime.com/high-profile/mike-lindell-attorney-facing-sanctions-over-ai-generated-motion-citing-cases-that-do-not-exist-in-defamation-case-against-ex-dominion-executive/ and more reports.) For this reason, it’s essential to verify the information and sources provided before relying on anything AI-generated.
In the past, I’ve used AI tools to verify suspect information, and they’ve failed – information I knew was fake was incorrectly identified as valid by them. I just rechecked one test – a spammy LinkedIn profile. Although AI still believed the profile was genuine, this time, red flags were highlighted. It’s highly encouraging if AI can detect human lies, especially since scammers are now likely to use AI to craft convincing phishing emails, as was potentially the case with the email I received.
Here are more details:
- Perplexity results: https://aware.tiny.us/nestlescamp including a question on whether the email could have been created using AI
- ChatGPT’s answer: https://aware.tiny.us/nestlescamc.
Copilot, Gemini, and Claude picked out similar elements.
Arthur Weiss has been an infopreneur for almost 30 years. He founded AWARE in 1995 after a career at the business information company Dun & Bradstreet. He specializes in competitive and marketing intelligence using open sources (OSINT). Recently, he has pivoted to new areas, including exploring how AI tools can support infopreneurs. His latest insights can be read in International Marketing & Competitive Intelligence and Computers in Libraries magazines. He may be contacted at a.weiss@aware.co.uk.
