When the open web starts training on itself The web is no longer a clean corpus of human writing with a sprinkle of machine output. It is a mixed substrate, and the mixture is getting denser every quarter. Articles, product descriptions, image-board uploads, customer reviews, and even academic preprints now arrive partly or wholly generated. The interesting question is not whether this happened — it has — but what it means for the next generation of models trained on what the previous generation produced. The short version: detection-only strategies are brittle, training pipelines need data-provenance discipline, and the durable path runs through cryptographic provenance standards like C2PA rather than through ever-cleverer classifiers chasing this week’s generator. In our work with content platforms and regulated-industry teams, the groups that combine detection with provenance build a defensible posture; the groups that depend on detectors alone discover, usually painfully, that their pipeline ages in weeks. What “AI-generated data” actually means in a training pipeline The phrase covers at least three distinct categories, and conflating them produces sloppy reasoning. First, there is direct synthetic output — text from large language models, images from diffusion models, audio from voice-cloning systems. These are the obvious cases. Detection research focuses here because the artefacts (token-level distributions, frequency-domain signatures, perceptual hashes) are most studied. Second, there is AI-assisted human content — a draft written by a model and edited by a person, a photo retouched with a generative fill tool, a translation produced by a neural system and reviewed by a human. This category is the largest in practice and the hardest to classify, because the final artefact carries signal from both producers. Third, there is derivative synthetic content — model output that has been re-summarised, paraphrased, or recompressed by other models or humans. After two or three hops the original generator’s fingerprints are largely gone, and statistical detectors collapse to chance. A pipeline that treats these three as a single bucket will overestimate its ability to filter the corpus. The model-collapse argument, stated precisely Model collapse is the term used in the literature for what happens when generative models are repeatedly trained on data produced by previous generations of generative models. The intuition is that each generation slightly narrows the distribution — the tails get clipped, rare patterns drop out, and over enough rounds the model converges on a low-entropy caricature of its original training distribution. This is an observed pattern in controlled experiments on small-scale generators; in our experience reviewing production training corpora, the practical risk is not full collapse but slow drift in the long tail. Specialist vocabulary, regional dialects, and minority code styles degrade first. The model still looks impressive on benchmark prompts because the head of the distribution is over-represented in both the synthetic and the human web. The damage hides in the tail. The mitigation is not exotic. It is data-provenance discipline at ingestion time: knowing what is human-authored, what is model-authored, and how much weight each category carries during training. Teams that cannot answer those questions about their own corpus are betting that the open web stays clean enough on its own. Why detection alone is a losing posture Detectors fall into a few broad families: Detector family What it measures Where it breaks Statistical classifiers (token entropy, perplexity) Distributional fingerprints of a specific generator family New generators, paraphrased text, short passages Perceptual hashing Near-duplicates of known synthetic assets Re-encoding, crop/rotate, mild generative edits Embedding-distance methods Similarity to a known synthetic-content embedding cluster Shifts when the generator updates; high false-positive rate on stylised human writing Watermark verification A signal the generator embedded on purpose Only works for cooperating generators; survives transcoding only sometimes Each family is useful in a narrow window and degrades quickly outside it. Independent evaluations of named consumer detectors — Winston AI, GPTZero, TruthScan and similar — have repeatedly reported false-positive rates that make them unsafe for high-stakes adjudication on their own; this is an observed pattern across multiple third-party audits rather than a single benchmark, and the specific numbers shift each time a major generator releases. The honest summary is that best-in-class detectors are useful as one signal in a stack and dangerous as a sole gatekeeper. The structural reason is straightforward: detectors are trained on yesterday’s generators, and generators are released continuously. A detector deployed in production at month zero is measuring something different by month six. Cryptographic provenance: the durable layer The alternative — or rather, the necessary complement — is provenance at the producer side. The Coalition for Content Provenance and Authenticity (C2PA) standard defines a way for capture devices, editing tools, and generative systems to attach signed metadata to assets describing who or what produced them and what edits have been applied. Adobe, Microsoft, OpenAI, and several camera manufacturers ship C2PA-compatible tooling today. Two honest caveats are worth stating directly. First, C2PA is opt-in at the producer; an adversary who controls the production pipeline can simply not sign, and an unsigned asset is not evidence of anything. Second, signatures cover the file as produced, not the pixels as they appear after a screenshot, a re-crop, or a social-platform transcode. Provenance survives cooperative workflows; it does not survive deliberate stripping. What provenance does well is raise the cost of adversarial generation enough to deter the casual case and create an auditable trail for the contested case. For a publisher, a content platform, or a regulated buyer of stock imagery, that is the right shape of defence. The asset either carries a chain of custody back to a known producer or it does not, and the absence of a chain is itself a signal that should change how the asset is handled downstream. A layered posture for teams that ingest mixed content The decision rubric we recommend to clients evaluating their content-authenticity needs: Producer-side provenance wherever you control the pipeline. If your platform generates or accepts uploaded images, attach or verify C2PA manifests. This is the lowest-effort, highest-durability layer. Detection as a triage signal, not a verdict. Use detectors to route content into review tiers, never to issue an automatic rejection or accusation. The false-positive cost on a human writer wrongly flagged is asymmetric. Hashing against known-synthetic corpora for the long tail of re-uploaded generated assets. Perceptual hashing catches the laziest cases cheaply. Editorial or human review for the top of the risk pyramid — anything that affects payment, identity, safety, or public-record claims. Training-corpus hygiene if you are training or fine-tuning models on web-scraped data. Treat unsigned, unverified content as a separate class from signed-and-verified content, and weight accordingly. This is a stack, not a silver bullet. The teams that get burned are the ones that picked one layer — usually a single off-the-shelf detector — and treated it as the whole answer. How does this connect to detecting AI-generated images specifically? Image detection is the most-studied surface because the artefacts are richest. Frequency-domain signatures, sensor-noise patterns, and diffusion-model fingerprints have all been used as classifier inputs. The honest assessment, which we develop in AI vs Real Images: How to Tell the Difference, is that classifier-based image detection works well within a generation of a known model family and degrades sharply across model updates. The provenance argument applies with extra force to images because cameras can sign at capture — a path text content does not have a direct equivalent for. What this means for the next two years Two things are likely. First, the proportion of public-web content that is model-touched will keep rising, and detectors will keep losing ground in raw-classification terms. Second, the institutions that care most about authenticity — newsrooms, courts, regulated industries, large platforms — will move toward provenance-based workflows because they are the only ones that compose with legal evidence standards. The teams we work with on generative-AI feasibility audits increasingly arrive with a detection-first question and leave with a provenance-first plan. That reordering is the practical content of this shift. FAQ How do current AI image detectors actually work — embeddings, watermarks, perceptual hashing, classifiers? Detectors combine several methods. Statistical classifiers look at frequency-domain or token-distribution fingerprints of known generator families. Embedding-distance methods place an asset in a learned representation space and measure similarity to known-synthetic clusters. Perceptual hashing catches near-duplicates of previously seen synthetic assets. Watermark verification checks for signals a cooperating generator embedded on purpose. Each is useful in a narrow window; none is robust across generator updates. Can C2PA cryptographic provenance be faked, and what is its real coverage in 2026? C2PA cannot be cryptographically forged without compromising a producer’s signing key, but it can be stripped — a screenshot or a transcode through a non-C2PA-aware tool produces an unsigned file. Coverage is real but uneven: major editing tools, several camera vendors, and the largest generative-AI vendors support it, but the long tail of consumer apps and social platforms does not yet preserve manifests end-to-end. The right framing is that a present manifest is strong evidence and a missing manifest is not, by itself, evidence of forgery. What is the failure rate of best-in-class detectors (Winston, GPTZero, TruthScan) on real content? Independent evaluations have repeatedly reported false-positive and false-negative rates high enough to make any single detector unsafe as a sole gatekeeper, particularly on short passages, paraphrased text, and non-native-English writing. Specific numbers move each generator release. The operationally relevant claim is not “detector X is wrong N% of the time” but “no current detector is reliable enough to be used without a human-review layer for any decision with real consequences.” Where does perceptual hashing fit in the detection stack alongside ML-based detectors? Perceptual hashing catches re-uploads and lightly-modified copies of previously known synthetic assets at very low cost. It is the right first layer for platforms dealing with high-volume re-circulation of viral synthetic content. It does not detect new generations of synthetic content; for that, classifier or provenance layers are required. How does an enterprise deploy a layered detection, provenance, and governance stack for AI content? Start with provenance at every point you control — capture, editing, generation, ingestion. Add detection as a triage signal to route content into review tiers. Use perceptual hashing for high-volume duplicate catching. Reserve human review for high-stakes content. Finally, instrument training pipelines to distinguish signed, verified, and unverified content classes and weight them differently. The order matters: provenance first, detection second, human review for the top of the risk pyramid. Which detection patterns work for images, text, audio, and video, and where do they break? Image detection is the most mature, with frequency-domain and diffusion-fingerprint methods working within a generator family. Text detection is the least reliable, especially on short or edited passages. Audio detection works moderately well against current voice-cloning systems but degrades when synthetic audio is mixed with real recordings. Video is the hardest because temporal consistency adds signal but compression strips it. In all four modalities, provenance is the more durable answer; detection is the bridging layer. Where TechnoLynx fits Our GenAI Feasibility Audit — defensive variant — evaluates a team’s content-authenticity needs against the detection-versus-provenance trade-off and produces a layered deployment plan rather than a single-tool recommendation. The output is a stack appropriate to the team’s actual risk surface, not a generic “use detector X” answer that will be obsolete by the next model release. Engineering note: most pipelines we audit fail not at the detector but at the ingestion boundary, where unsigned and signed assets are commingled before any classification runs. Provenance lost at ingestion cannot be recovered downstream.
