As the current artificial intelligence wave is cresting, negativity is setting in as scepticism over what the “change everything” technology will actually deliver grows stronger. Even if you adjust for humanity’s understandable bias for thinking the worst, just in case it comes true, the amount of Cassandras warning that AI is overblown and even unsafe easily outweighs the positive takes.
Some of the caution comes from unexpected quarters, like Google, the tech giant that has gone in boots and all with AI at every level, from handsets to cloud services, to shore up its business fortunes.
Researchers at Google’s DeepMind have published a thought-provoking paper which describes real-world misuses of generative AI (GenAI); this is the technology that can create text, digital images, videos and audio.
That’s because GenAI’s large language models (LLMs) are trained on massive amounts of human generated data that pattern-recognition algorithms running on powerful computer systems assemble into realistic renditions that can be eerily indistinguishable from what humans are capable of producing. This is the interactive form of AI that many people are exposed to through chatbots like Google Gemini, OpenAI ChatGPT, Microsoft Copilot and Anthropic Claude.
To nobody’s surprise, such advanced digital plagiarism of human works and characteristics have opened up AI to vast possibilities for abuse, as the Google researchers’ paper Generative AI Misuse: A Taxonomy of Tactics and Insights from Real-World Data shows.
The paper describes a plethora of tactics for impersonating real people and their work for malicious purposes online with AI, and it’s a must-read for anyone trying to understand where we are headed with the technology.
What stands out is that the abuse does not come from clever prompt hacking to get past AI guardrails, but by simply using the systems’ easily accessible capabilities, the DeepMind researchers note.
That is, it entails using AI as intended, with minimal technical expertise. It enables malicious people to engage in online fraud, sockpuppeting (yes, that’s a word) for amplification to manipulate opinion or falsely inflate popularity, impersonate celebrities for fake ads, creating bogus websites to trick people into downloading malware, sharpening up phishing campaigns and much more.
Anyone working in the information security field will “head desk” while reading the Google DeepMind paper, wondering how they’ll be able to defend against machine generated attacks that will be launched at scale as threat actors adopt AI.
Companies aware that AI can go off the guardrails
Misuse of AI is definitely something that publicly traded tech companies such as Facebook’s parent company Meta, Microsoft, Google, Oracle and others are aware of. To the point that they have started adding the AI threat scenarios to the risk sections in their mandatory investment disclosure documents.
Fun disclosure from Google: “Unintended consequences, uses or customization of our AI tools and systems may negatively affect human rights, privacy, employment or other social concerns." https://t.co/lyjWJjHLE6
— Alex Weprin (@alexweprin) March 28, 2024
Sometimes it’s not third party threat actors that pose the AI risk, but the organisations building the technology themselves. Germany’s Large-Scale Artificial Intelligence Network - LAION - is a non-profit that has assembled some of the most popular free datasets in the world, sponsored by AI companies such as Hugging Face and Stability AI.
LAION says its datasets like the 5.85 billion image-text pair 5B one “are simply indexes to the Internet” which link to pictures. United States-based Human Rights Watch took a closer look and found that the dataset led to personal photos of Australian children being scraped off the web, and used to train AI models.
HRW found that the LAION-5B dataset “contains links to identifiable photos of Australian children. Some children’s names are listed in the accompanying caption or the URL where the image is stored. In many cases, their identities are easily traceable, including information on when and where the child was at the time their photo was taken.”
Such datasets could be misused by other tools to create deepfakes, putting children at risk, HRW technology researcher Hye Jung Han pointed out. Interest.co.nz asked Han if New Zealand children were featured in the data set too. Han said “I wouldn’t be surprised” but couldn’t confirm if this was the case as she didn’t check for NZ kids.
LAION is aware that its data can lead to illegal content, and took down 5B in December for a safety review.
More data, more power, more money, but for what?
AI requires a constant stream of new data to update the models that generate content, like a zombie insatiably hunting for fresh brains. Tech companies think that means they can just take it and then sell it back to us through GenAI service subscriptions via their proprietary billion-dollar cloud AI systems.
Recently, Microsoft’s head of AI, Mustafa Suleyman created a furore by claiming web content is “freeware” that companies can help themselves to. Needless to say, Suleyman might need to take a refresher course in intellectual property law and precedents before he broaches the AI data gathering subject again.
A technology that can be seriously abused, and which can really damage an organisation’s reputation; in fact, AI might not even work as well as existing tools, because its output is an algorithmic guesstimate of what might appeal to users.
Twitter users were given an example of well-known interface design tool Figma (which Adobe tried to buy but wasn’t allowed to by European Union regulators) producing a weather app.
Figma AI looks rather heavily trained on existing apps.
— Andy Allen (@asallen) July 1, 2024
This is a "weather app" using the new Make Designs feature and the results are basically Apple's Weather app (left). Tried three times, same results. https://t.co/Ij20OpPCer pic.twitter.com/psFTV6daVD
Investment bankers Goldman Sachs took aim at GenAI in its June 2024 Top of Mind research report, saying tech companies are set to spend over US$1 trillion in capital expenditure on the technology in coming years, “with so far little to show for it”.
Long story short, the experts Goldman Sachs spoke to think that AI is fine for simple coding tasks as it's been extensively trained on that kind of material. More complex tasks? Not so much. Also, AI technology costs too much to develop.
As Jim Covello, the head of global equity research at Goldman Sachs puts it: “What US$1 trillion problem will AI solve? Replacing low-wage jobs with tremendously costly technology is basically the polar opposite of the prior technology transitions I’ve witnessed in my thirty years of closely following the tech industry.”
Even the more AI-enthusiastic analysts at Goldman Sachs note that we have yet to work out what the technology’s killer application is. Plus, AI looks set to be held back by its enormous energy consumption and chip shortages, assuming that throwing more hardware at the technology would improve it, which too is far from certain.
AI isn’t actually that new, although you might think so given the massive hype in the last couple of years that stems from the generative variant of the technology. That AI can be a very useful tool for specific applications is beyond doubt. However, thanks to GenAI being the loose cannon it is, a blackhole for investment and resources that doesn’t seem to yield much in terms of productivity increase and savings, people are starting to view the technology as a disappointment.
“Bullshit” even, as University of Glasgow’s straight talking researchers Michael Townsen Hicks, James Humphries and Joe Slater put it in their very accessible paper. And yes, people are already asking if GenAI is a bubble waiting to burst.
We welcome your comments below. If you are not already registered, please register to comment
Remember we welcome robust, respectful and insightful debate. We don't welcome abusive or defamatory comments and will de-register those repeatedly making such comments. Our current comment policy is here.