Beware of websites claiming they can detect faked AI generated content; they might be fakes themselves

Artificial intelligence (AI) is changing just about everything, but often in an unexpected fashion. Recently, an email arrived that looked like your run of the mill search engine optimisation spam: "could you insert a link to a site in a story?"

Normally, those messages are marked as junk and deleted on sight, but this time round I looked at the details of it.

The message was from someone allegedly working for AI detection site GPTZero. It bills itself as:

More than an AI detector
Preserve What's Human
Since inventing AI detection, GPTZero incorporates the latest research in detecting ChatGPT, GPT4, Google-Gemini, LLaMa, and new AI models, and investigating their sources.

GPTZero is a name that I've seen fly past, and the site has lots of well-known media logos on the front page.

However, the message I received seemed off. The mail headers look correct and it passed anti-spam measures, so it probably was sent from GPTzero's mail server. 

The sender was just a first name, "Meghan" and ran the text through GPTZero which said there was a 68% likelihood of it being written by an AI. Hmm.

"Meghan" wrote:

"We value the importance of providing readers with useful resources, and I wanted to ask if you could kindly include a link to our website [gptzero.me/] where you mention AI detectors. There are several copycat websites and potential scams, so including this link will allow your readers to easily access our software."

Well that looked weird. Searching on GPTZero found a bunch of sites with that domain name, in different top level domains:

• GptZero.cc
• GptZero.dev
• GptZero.Me
• GptZero.tools

Then there are variations on the domain name, like:

• Gpt-Zero.com
• Gpt-Zero.org
• Gpt-Zero.cc 

In fact, the story that "Meghan" asked me to insert a link into referred to a site with a slightly different but similar name, ZeroGPT.

Many of the sites I looked at don't provide any clues as to who runs them. No contact details, anonymised domain name registration details, site behind Cloudflare's proxy servers and so forth. 

Even if they do have contact details and pictures of the happy humans running the site, it's dead easy to fake those things with... AI. This includes creating realistic pictures of humans and their LinkedIn profiles, and more.

I think "Meghan" is right. There are indeed "several copycat websites and potential scams". Search for "AI detection" and you'll find even more. 

They don't seem to work all that well either, possibly not even using AI to detect machine generated text. Some of them mark direct quotes that I know were said by a person as AI generated. 

To see what "Meghan" and the GPTZero team had to say about that, I sent separate emails to them. After several days of patiently waiting, there have been no responses - yes, I've checked the spam folder.

Ah well. This is yet another 2024 story which illustrates how much you can trust black box technology, that you cannot tell how it works, like AI. 

Meanwhile, do not enter anything sensitive into an "AI detector" site whether you pay for it or not, and be very sceptical of the output. The Internet is full of enterprising scammers that take advantage of people trusting computers way too much.

Update There is research published on how well the "AI detector" sites work, including this by Debora Weber-Wulff, Alla Anohina-Naumeca, Sonja Bjelobaba, T. Foltýnek, J. Guerrero-Dib, Olumide Popoola, Petr Sigut, and Lorna Waddington. From the TL;DR:

"The researchers conclude that the available detection tools for AI-generated text are neither accurate nor reliable and have a main bias towards classifying the output as human-written rather than detecting AI-generated text."

Hat tip: Graham Cluley and Mark Stockley.