Days after visiting pages with artificially generated photos – without liking or commenting – the Facebook algorithm started recommending other pages that use AI
*By Renee DiResta
If you’ve spent any time on Facebook in the last 6 months, you may have noticed realistic images that are too good to be true: children holding paintings that look like the work of professional artists, or majestic cabin interiors that are the stuff of Airbnb dreams.
Others, like the crustacean depictions of Jesus, are simply bizarre.
Just like the image of the pope With a coat that went viral in May 2023, these AI-generated images are increasingly common – and popular – on social media. Although many of them border on surrealism, they are often used to attract users’ attention.
Our team of researchers from Stanford University Internet Observatory It’s from Georgetown University Center for Security and Emerging Technologies analyzed more than 100 Facebook pages that published large volumes of AI-generated content. We publish the results in March 2024 as a article initial, which means that the conclusions have not yet been reviewed by the evaluators.
We explored image patterns, discovered signs of coordination between some of the pages, and tried to discern the likely goals of the authors behind the posts.
The page operators appeared to be publishing AI-generated images of babies, kitchens, or birthday cakes for a number of reasons.
There were content creators who harmlessly sought to increase the number of followers with synthetic content; scammers who used pages stolen from small businesses to advertise products that appeared to not exist; and spammers who shared AI-generated images of animals and sent users to ad-filled sites, allowing owners to earn advertising revenue without creating quality content.
Our findings suggest that these AI-generated images appeal to users – and Facebook’s recommendation algorithm may be organically promoting these posts.
Generative AI Meets Scams and Spam
Internet spammers and scammers are nothing new.
For more than two decades, they used bulk emails unsolicited to promote pyramid schemes. They target senior citizenspretending to be brand representatives or IT technicians.
On social media, profiteers have used clickbait articles to drive users to ad-laden sites. Remember the 2016 US presidential election, when Macedonian teenagers They shared sensational political memes on Facebook and collected advertising revenue after users visited the URLs they posted. Teenagers didn’t care who won the election. They just wanted to win money.
In the early 2010s, spammers captured people’s attention with ads that promised anyone could lose belly fat or learn a new language with “a strange trick“.
AI-generated content has become another “weird trick”.
It is visually appealing and cheap to produce, allowing scammers and spammers to generate large volumes of interesting posts. Some of the pages we observed uploaded dozens of different images per day. In doing so, they followed the advice of the Goal itself for page creators. The company suggests that publishing frequently helps creators get the kind of algorithmic collection that leads to their content appearing on “Feed“, formerly known as “News feed“.
Much of the content continues to be, in some ways, clickbait: The “Jesus Camaro” makes people stop to look and inspires shares just by being so bizarre.
Many users react by liking the post or leaving a comment. This indicates to network mediators that perhaps the content should be placed in the feeds of even more people.
Some of the more established spammers we observed, likely knowing this, have improved their engagement by moving from publishing URLs to publishing AI-generated images. They then commented on the publication of the AI-generated images with the URLs of websites full of ads that they wanted users to click on.
But mainstream creators have also capitalized on the involvement of AI-generated images without obviously violating the platform’s policies.
Rate “my” work
When we looked for the post captions on CrowdTangle – a social media monitoring platform owned by Meta that will be shutting down in August – we discovered that they were “copypasta” captions, meaning they were repeated across every post.
Some of the copypasta captions encouraged interaction, directly asking users to, for example, rate a “painting” from a debut artist – even when the image was generated by AI – or wishing an elderly person a happy birthday. Facebook users frequently responded to the AI-generated images with encouraging and congratulatory comments.
Algorithms promote AI-created content
Our research visibly altered our own Facebook feeds: within days of visiting the pages – and without commenting, liking or following any material – the Facebook algorithm recommended huge numbers of other AI-generated content.
Interestingly, the fact that we saw groups of, for example, AI-generated miniature cow pages did not lead to a short-term increase in recommendations for pages focused on real miniature cows, full-sized cows, or other farm animals. Instead, the algorithm recommended pages on a range of topics and themes, but with one thing in common: they contained AI-generated images.
In 2022, the technology website The Verge detailed an internal Facebook memo about proposed changes to the company’s algorithm.
The algorithm, according to the memo, would become a “discovery engine“, allowing users to contact posts from individuals and pages they have not explicitly searched for, similar to the “For you” from TikTok.
We analyze the “Widely Viewed Content Reporting” from Facebook, which lists the most popular content, domains, links, pages and publications on the platform per quarter.
It showed that the proportion of content users saw from pages and people they don’t follow increased steadily between 2021 and 2023. Changes to the algorithm gave more space for AI-generated content to be organically recommended without prior engagement – which perhaps explains our experiences and those of other users.
“This post is a gift from AI”
Since Meta does not currently flag AI-generated content by default, we have sometimes observed users warning others about scam or spam AI content with infographics.
However, Meta appears to be aware of the potential problems if AI-generated content blends into the information environment without warning. The company did several adverts about how you plan to handle AI-generated content.
In May 2024, Facebook will begin enforcing a “Made by AI” to content that it can reliably detect as synthetic.
But the devil is in the details. How accurate will the detection models be? What AI-generated content will go unnoticed? What content will be marked incorrectly? And what will the public think of these labels?
While our work has focused on spam and Facebook scams, there are broader implications.
The reporters wrote about AI-generated videos aimed at kids on YouTube and influencers on TikTok using generative AI to get profits.
Social media platforms will have to think about how to treat AI-generated content; It’s certainly possible that user engagement will decline if online worlds become filled with artificially generated posts, images and videos.
Shrimp Jesus could be an obvious fake. But the challenge of evaluating what is real is yet to come.
*Renee DiResta is a principal investigator at Stanford University’s Internet Observatory; Abhiram Reddy is a research assistant at the Center for Security and Emerging Technology at Georgetown University; Josh A. Goldstein is a guest researcher at the Center for Security and Emerging Technology at Georgetown University
Translated by Luciana Saravia. Read the original in English.
O Power360 has a partnership with two divisions of Nieman Foundationin Harvard: O Nieman Journalism Lab and the Nieman Reports. The agreement consists of translating into Portuguese the texts that Nieman Journalism Lab and Nieman Reports publish and publishing this material on Power360. To access all translations already published, click here.
#religious #images #fake #portraits #form #spam