AI does three jobs in social media. It ranks the posts in your feed by how likely you are to engage with each one. It screens content for policy violations and sends the cases it cannot settle to human reviewers. And generative AI tools are increasingly deployed to help create the posts themselves. The first two jobs run on almost every major platform. The third is newer, and one controlled experiment found a trade-off: some generative tools raised engagement and posting volume but lowered perceived quality and authenticity. Brands meet all three at once, so we would learn how each works before deciding how far to lean on it. For the wider picture of how AI is used across marketing functions, our AI in marketing overview sets out the main categories.
Why does my social media feed feel so personalised?
Your feed feels personalised because it is built from predictions: for each post you could be shown, a model estimates how likely you are to interact with it, and the feed is mostly those posts sorted from highest score to lowest. Most recommendation algorithms answer one question: how did users similar to this user engage with posts similar to this post? The Knight First Amendment Institute’s essay on recommendation algorithms notes that this approach is popular because it has repeatedly worked well in practice.
Each platform defines “engagement” in its own way, and that definition shapes the feed:
- Facebook optimises for “Meaningful Social Interactions”, a weighted average of likes, reactions, reshares and comments.
- YouTube optimises for expected watch time, meaning how long the model predicts a video will be watched.
- TikTok combines predicted probabilities of liking, sharing, commenting on or skipping a video into one overall score per video.
- X (formerly Twitter) ranks with a neural network of roughly 48 million parameters that outputs ten engagement probabilities per post, according to its 2023 engineering blog.
The funnel behind a ranked feed
Ranking at this scale is usually split into stages, because running the heaviest model on every candidate is not feasible. Meta’s engineering description of Instagram Explore shows this pattern. Candidates are retrieved first. A lightweight first-stage ranker then scores thousands of them, and a heavier, more precise model works only on the best 100. The final order comes from a “value model”, a weighted sum of predicted probabilities such as P(click) and P(like), minus a weighted P(see less). After that, rules filter or downrank items on integrity scores and reshuffle for diversity, for example so that the same author does not appear several times in a row. TikTok runs a similar final check and replaces top-ranked videos that are too alike (the same sound, for instance) to keep the feed varied.
The weights are set by hand, and they have consequences. In Facebook’s interaction formula, a reshare was once weighted 30 times higher than a like, and in 2020 the company cut that weight to 1.5 after recognising unintended effects. Georgetown’s Knight-Georgetown Institute put it plainly in its March 2025 Better Feeds report: borderline content clustering at the top of feeds “is not an inevitable outcome but rather a choice of design”.
Why personal predictions still miss so often
Recommendation models are imprecise at the level of individual users. The Knight Institute essay puts the engagement rate on recommended posts below 1% on most platforms, and only a little over 5% even on TikTok, the outlier. The essay argues that these systems are imprecise for any one user but accurate in aggregate: they are good at spotting what will go viral and at matching niche content to the small audience that wants it. Feeds also deliberately mix in unfamiliar material, because recommending only what you already like stops the model learning whether you would like something new. On X, the For You timeline averages 50% posts from accounts you follow and 50% from accounts you do not.
What ranking does to what people believe
Ranking changes what people see, and a randomised experiment shows it can shift what they think. In a randomised experiment with active US users of X over seven weeks in summer 2023, switching from the chronological feed to the algorithmic one raised engagement by 0.14 standard deviations and moved political opinion towards more conservative positions. Switching the algorithm off produced no measurable change in attitudes, which the authors partly explain by users continuing to follow accounts they had discovered through it. An earlier study of Twitter’s home timeline found that the mainstream political right received more algorithmic amplification than the mainstream left in 6 of 7 countries studied. Against a widespread assumption, the same study found no evidence that far-left or far-right groups were amplified more than moderate ones.
EU regulation now covers ranking. Article 27 of the EU Digital Services Act requires platforms to state the main parameters of their recommender systems in plain language and to explain why certain information is suggested. Where users have several ranking options, the platform must let them choose and change their option directly from the part of the interface where ranking happens. Very large platforms, those with over 45 million EU users, must also offer at least one recommender option that is not based on profiling. Georgetown’s review of these disclosures found that every platform says it processes engagement data, and the two that disclose any weighting (TikTok and Snap) say engagement signals carry the most weight.
For commercial content creators, the practical consequence is volatile reach. If one post is 10% more engaging than another, the algorithm compounds that difference over time, and the Knight Institute essay suggests the first could end up with a hundred times the reach. We would plan social output around that variance rather than treating any single post’s reach as a measure of its quality.
How does AI moderate content on social media platforms?
AI moderates content by triage. Automated classifiers decide whether a post stays up, comes down or goes to a human reviewer. Platforms combine human moderators with rule-based techniques and machine-learning models, and the Oversight Board noted in 2024 that most moderation decisions are now made by machines rather than people. Meta reports that more than 90% of the violating content it removes from Facebook and Instagram is detected by AI. It also states that hate speech prevalence on Facebook was 0.01-0.02% of content views as of Q3 2023, which is Meta’s own estimate.
Removal is only one of the tools. Platforms also demote content, showing it to fewer people without deleting it, or attach warning labels that inform rather than remove. TikTok removes content that breaks its Community Guidelines and makes content unsuitable for a broad audience ineligible for recommendation. YouTube says that demoting borderline content from 2019 onwards cut watch time on recommended, non-subscribed borderline videos in the US by 70%. This is why moderation and ranking now overlap: a classifier’s hate-speech score can become a ranking feature in its own right.
Human reviewer time is a fixed supply, so review jobs are prioritised. Research describing Facebook’s moderation system prioritises review jobs by predicted views, severity and review effort. Content that spreads fast gets attention first, because a violating post can reach a large audience before anyone removes it. Meta has also begun using large language models trained on its Community Standards, including to clear content out of review queues when it is highly confident the content does not break the rules.
Where AI moderation breaks down
On ordinary toxic inputs, commercial moderators look strong. On inputs designed to defeat them, they do not. A study of five commercial moderation products (from Google Cloud, Amazon Web Services, Baidu, Tencent and Alibaba, in the versions current in November 2022) measured an average detection rate of 97.8%. The same study then generated adversarial toxic test cases, and up to 100% of the image cases and 94.1% of the video cases bypassed the software and research models tested. Audio was weaker than text: 94.7% of toxic text-only videos were flagged, against 84.2% when the same sentences were spoken.
Other failures come from missing context. Model performance can drop sharply on examples that differ from the training data, and models lack training data for particular regions and local dialects. The Oversight Board has said Meta’s automated tools “time and again fail to account for context, particularly in languages other than English”. Coded language (“unalive”, deliberate misspellings, emojis) is built specifically to slip past automated detection. In one of the Board’s early cases, a nudity classifier removed a Brazilian breast cancer awareness post, even though the words “Breast Cancer” appeared on the image in Portuguese.
Where human review is still needed
The evidence points to four places where humans stay in the loop:
- Ambiguous content. Deciding whether someone is being bullied can be so nuanced and contextual that Facebook still sends it to human reviewers.
- Validating the classifiers. Human reviewers check borderline content and the accuracy of classifiers on samples of what they flag. YouTube’s “authoritative” and “borderline” classifiers rely on human evaluators who follow published rating guidelines.
- Legal requirements. The SoK survey of content moderation cites the EU General Data Protection Regulation among legislation that mandates human involvement.
- Appeals. In February 2024, Meta received more than seven million appeals against hate-speech removals, and eight in ten appellants chose to add context to their appeal.
The Oversight Board’s position is a fair summary: automation is essential given the volume, but enforcement that relies on automation alone over-enforces where context matters.
Can AI reliably generate finished social media content?
Some AI tools raise the volume of content people produce. But readers rated that output no better than unassisted writing, and rated comments from two of the four tools lower. In a controlled experiment with 680 US participants in a realistic social media environment, published by Nature, some AI writing tools built on GPT-4o increased engagement and the amount people posted. Every AI treatment also increased dislikes, and participants described the AI-assisted content as “robotic” and “generic”. None of the AI tools improved readers’ perceptions compared with the control group, and no single tool improved the experience of both writers and readers.
The damage also spread beyond the AI-assisted posts. The authors found that AI content lowered the quality of later conversation in a thread, including among people who were not using AI themselves. At the extreme end sits “AI slop”: generative content seen as lacking effort, quality or meaning, usually produced in high volume to make money or to deceive. Image and video slop spreads on Facebook and TikTok partly because it earns revenue for its creators.
We would treat generative output as a draft that a person shapes, checks and owns before it goes out. The study’s participants pointed the same way: they valued AI for ideas and for getting started when they were “lost for words”, and they asked for output that matched their own voice and context.
How can you tell if a social media post was made with AI?
Platform labels and embedded metadata help, though neither catches everything, and over half of US adults are not confident they can tell. In a Pew Research Center survey of US adults in June 2025, 76% said it is extremely or very important to be able to tell whether content was made by AI or by people, yet 53% were not too or not at all confident they could. An experiment shows how unreliable readers’ judgements are. In the Nature-published study, other users suspected 13.8% of participants in the no-AI control group of using AI, against 37.7-43.6% in the AI groups. Readers picked up the difference, but they also flagged people who had used no AI at all.
| Signal | What it tells you | Its limit |
|---|---|---|
| Platform label (for example Meta’s AI labels on Facebook, Instagram and Threads) | The platform found industry-standard indicators that the image is AI-generated | Not all AI content can be identified, and invisible markers can be stripped out |
| C2PA Content Credentials | Each recorded edit can be attributed, and an action performed by an AI/ML system is identified through the digitalSourceType field |
Provenance shows origin and history, not whether the content is true or accurate |
| TikTok AIGC label | TikTok says it has labelled over 3 billion videos using Content Credentials, creator labelling tools and invisible watermarking | The figure counts labelled videos, not how much AIGC goes unlabelled |
| EU AI Act Article 50 marking | Providers of generative systems must mark outputs in a machine-readable, detectable format, and deployers must disclose deepfakes | Providers’ marking solutions must be effective and robust only “as far as this is technically feasible” |
| Your own judgement | “Generic” or “robotic” tone is something readers do notice | Readers also label human-written posts as AI |
Labels are not a complete answer either. The Oversight Board has argued that labelling deepfake intimate imagery is not enough, because the harm comes from the images being shared and viewed, not only from people being misled about whether they are real. Detection, provenance and the longer-run effect of synthetic data on the web are covered in more depth in our piece on AI-generated data and internet quality.
Is AI replacing human customer service on social platforms?
None of the sources we used measures how far AI has replaced human customer service on social platforms, so we cannot give a figure. What the evidence does settle is the rule for the interaction. Under Article 50 of the EU AI Act, AI systems that interact directly with people must be designed so that people know they are dealing with an AI, unless that is obvious from the context.
The experimental evidence on AI-assisted replies counsels caution. Participants rated replies to their own comments lower when those replies came from the AI-assisted groups, and they described the AI responses as impersonal. In our view, AI fits best at the front of a service queue: drafting, triage and routine answers, with a clear handover to a person for complaints and edge cases. Moderation likewise pairs automation with human review where context matters.
Pros and cons of AI in social media, weighed on evidence
The benefits are documented. Algorithmic recommendation excels at niche content. The Knight Institute essay gives the example of a TikTok user in a small town being shown local restaurant videos from accounts that mostly post about other things. Being able to go viral has also let musicians and entertainers build an initial livelihood without traditional gatekeepers. Generative tools lower the barrier to taking part in a conversation. And given the volume of content, automated detection is essential to moderation at all.
Concerns about several costs centre on what the systems optimise for. Experimental studies cited by Georgetown found that, compared with alternative designs, engagement-based ranking raises anger, sadness and hostility towards outgroups, and increases the share of items expressing that negativity. Optimising for predicted engagement does not always match what users say satisfies them. Facebook found that showing more notifications raised engagement in the short term but had the opposite effect over a year. Automation can also amplify bias that is built into training data and system design.
Georgetown argues that the choice is not binary between engagement-optimised and chronological feeds. Its review concludes that recommender systems built to promote values other than engagement are both viable and successful at promoting user satisfaction. It argues that the main reason such designs are rare is misaligned incentives, not technical barriers.
How brands use AI in social media marketing
One brand use is generative ad creative inside the platforms’ own tools. Meta’s advertiser tools can generate full image variations of an ad, not just new backgrounds, along with alternative headlines and primary text, and Meta says it is testing copy that matches a brand’s voice. Casetify reported that Meta’s background generation feature raised its return on ad spend by 13%. That figure comes from one advertiser, reported through the vendor, and is a vendor claim rather than an independent test. The same tools can also produce images of product variants that do not exist, which Meta acknowledges as a possible use.
Audiences can push back hard. Coca-Cola’s AI-generated holiday commercials in November 2024 drew immediate backlash from viewers and artists. Activision’s generative ads for fake games on Facebook and Instagram were labelled AI slop by many. Read alongside the experimental evidence, our judgement is that the risk for a brand is less that AI content looks fake and more that it reads as generic in a feed that rewards distinctiveness. As our piece on AI in advertising puts it, the real question is where AI changes the economics of a marketing function and where it “just adds a faster way to produce content nobody reads”.
For brand work beyond the platforms’ built-in tools, TechnoLynx sets out its AI engineering for marketing.
One question remains open, and brands should watch it. The Nature-published experiment captured short-term reactions only, and its authors note that negative perceptions might fade with familiarity, or that familiarity might simply mean people pay less attention. Until longer studies settle it, the evidence stands: ranking rewards engagement, AI tools can raise content volume, and none improved how readers rated that content.
FAQ
Is TikTok a form of AI?
TikTok is a social platform, and its For You feed is powered by a recommendation system that ranks videos by predicted interaction scores. The Knight Institute describes TikTok as a demonstration of an almost purely algorithm-driven platform.
How is AI changing our society?
Pew Research Center’s June 2025 survey found that 50% of US adults are more concerned than excited about the growing use of AI in daily life, up from 37% in 2021. Among those who rate AI’s risks as high, 18% gave as their main reason that it makes it harder to tell what is real from what AI created.
