Alarm raised over bizarre images circulating on Facebook

Shrimp Jesus, grandmothers with cakes and suspiciously cozy log cabins have all led to alarm

(AFP via Getty Images)

Experts and users have warned about an increasing amount of seemingly AI-generated images, spreading across Facebook and other social networks.

The problem means that many groups are now filling with images intended to evoke a certain feeling: particularly wholesome images of grandmothers with cakes, for instance, or cozy cabins. Some are more unusual, such as a picture of Jesus made of crustaceans.

The problem has become sufficient that a team of researchers from  Stanford Internet Observatory and Georgetown University’s Center for Security and Emerging Technology have launched an investigation of more than 100 Facebook pages that had a particularly high amount of AI-generated content.

In their new paper, which has not yet been peer reviewed and is titled ‘How Spammers and Scammers Leverage AI-Generated Images on Facebook for Audience Growth’, they argue that the problem is a result of people looking to increase their following on social networks.

“Much of the research and discourse on risks from artificial intelligence (AI) image generators, such as DALL-E and Midjourney, has centered around whether they could be used to inject false information into political discourse,” the authors write. “We show that spammers and scammers - seemingly motivated by profit or clout, not ideology - are already using AI-generated images to gain significant traction on Facebook.”

The Independent asked Chris Cox, Meta’s chief product officer who built the original news feed, about the growth of such fake images during an artificial intelligence event earlier this month.

He said that “labelling is absolutely critical”. He pointed for instance to new rules that require every post in a feed to include a message indicating that it was generated by artificial intelligence, whether that was using one of Meta’s own AI systems or those from other companies, such as Midjourney.

“The key here is: user experience along the way, making sure nobody’s surprised about the experience they’re having, and relying on simple user experience tools that label AIs when we see it,” he said. “For what it’s worth, the amount of AI generated content in feeds is still de minimus.”

But the new paper suggests that for now at least some of those groups are sharing unlabelled images. Those unlabelled images even appear to people who do not follow the groups or realise that they are AI-generated, meaning that they could easily lead to people confusing them for genuine pictures.

When that happens, those images often receive a flurry of supportive comments. But it is unclear how many of those comments are themselves being written partly or wholly using artificial intelligence tools.

That has led to some fear of the “dead internet” – the idea that the web will soon be filled with automated systems both creating and reacting to content, and thereby making many websites increasingly hostile to human users.