An audit-evidence pack is not a deck someone writes when a regulator calls. It is a fixed set of sections, each populated by the moderation pipeline while decisions are being made, and each answerable down to one item on one date. Five sections carry the weight: policy-to-prompt mapping, per-decision adjudication records, version pinning, escalation and appeal history, and sampling evidence for automated-only actions.
The reason to know the section list before you deploy is blunt. Sections determine fields, fields determine logging, and logging has to exist at decision time. A summary written afterwards can describe how the system generally behaves; it cannot be queried for what happened to one specific piece of content on 14 March. That distinction is the whole engineering argument, and it is why we treat pack anatomy as a pipeline requirement rather than a documentation task.
What does a content-moderation audit-evidence pack contain, section by section?
| Section | What it holds | Populated | Owner |
|---|---|---|---|
| Policy-to-prompt mapping | Each policy clause with a stable identifier, and the prompt, rule, or threshold that encodes it | Fixed structure; revised when policy or prompts change | Policy + ML jointly |
| Per-decision record | Content reference, clause invoked, score and threshold, action taken, reviewer identity or automated flag, timestamp | Continuously, at decision time | Moderation pipeline |
| Version pinning | Model version, prompt revision, policy revision, reviewer-guidance revision attached to each decision | Continuously, at decision time | Platform engineering |
| Escalation and appeal | Queue path, escalation triggers, appeal submission, adjudication outcome, notice given to the user | Continuously, on event | Trust and safety operations |
| Sampling evidence | Review sample design, sample size, error findings for actions taken without human review | Periodically, per sampling cycle | Quality assurance |
Everything above is queryable. Nothing above requires a narrative to be written after the fact. That is the test a pack section has to pass before it earns a place in the list.
Which fields have to be captured at decision time?
The fields that cannot be reconstructed later are the ones that describe a system state that no longer exists. Model version is the obvious case — a classifier retrained twice since March cannot tell you what its predecessor scored. Prompt revision and policy revision behave the same way. So does the reviewer-guidance text that the human adjudicator was reading when they made the call, and the threshold that was in force at the moment the score crossed it.
Score, action, and timestamp are usually logged already. The pinning fields usually are not, because nobody needed them until an inquiry landed. Retrofitting them across a pipeline in production is the expensive path, and it is the specific cost the section list exists to make visible before deployment rather than during an inquiry.
Reading the mapping section without engineering help
The policy-to-prompt mapping is the section most likely to be read by someone who does not write code. It works when it is a table, not a diagram: one row per policy clause, with the clause identifier, the clause text as published, the prompt or rule text that encodes it, the threshold, and the revision date of each. A reviewer following a decision back to its clause should be able to do it by reading two columns.
What breaks this section is drift between the published policy wording and the operational encoding. Both change, rarely on the same day. Carrying the revision date on each side is what lets someone reading the pack in November see which pairing was live in March. Our approach to documenting how AI decisions are made and governed treats this dual dating as non-optional, because without it the mapping is true only for today.
Automated-only decisions versus human-reviewed ones
The per-decision section has two shapes, and confusing them is a common defect.
A human-reviewed decision carries an adjudication trail: which queue the item entered, who adjudicated it, against which guidance revision, what they decided, and how long it took. The reviewer identity may be a pseudonymous operator ID rather than a name — the pack needs accountability, not personal data.
An automated-only decision has no adjudicator, and pretending otherwise is worse than admitting it. Its record carries the score, the threshold, the pinned versions, and a pointer into the sampling section. Sampling is what stands in for per-item human judgement at scale: a documented sample design, a sample size, and the error rate found when humans re-checked the sampled decisions. Defensibility for automated-only actions rests on sample design, not on classifier accuracy — an aggregate precision figure describes the model, while sampling evidence describes the enforcement.
Named tooling matters here mainly as plumbing. Version pinning is a natural fit for whatever registry already tracks model artefacts — MLflow, a container digest in a Kubernetes deployment manifest, or a git SHA for prompt files. The pack does not care which, provided the identifier recorded on the decision resolves to something immutable. A tag that can be moved is not a pin.
Fixed structure, per-inquiry contents
Two things get confused when teams first build a pack. The section list is invariant — the same five sections serve every inquiry, every year. The contents are drawn per inquiry, filtered to the decisions, dates, and policy clauses in question.
That split is what lets the pack survive policy revisions without a rewrite. A new prohibited-content category adds rows to the mapping section; it does not change the anatomy. We look at this in more depth alongside the wider question of how moderation decisions are made defensible from policy through to per-decision evidence, where the pipeline instrumentation behind these sections gets its full treatment.
The practical payoff is retrieval speed. When each section has a known owner and a known data source, an inquiry becomes a lookup measured in days rather than a reconstruction project measured in weeks (an observed pattern across governance engagements, not a benchmarked figure — the actual duration depends heavily on log retention windows).
Where the pack stops
Every section here documents operational workflow: how a written policy was applied to a piece of content, by which version of which system, and who adjudicated. None of it adjudicates whether the policy itself is correct, and none of it tracks user behaviour beyond the specific items acted on. That boundary is easy to erode. A section that starts recording patterns across a user’s history has stopped being enforcement evidence and become something else, with a different legal footing and a different review process.
Keeping the anatomy narrow is what keeps it defensible. The pack answers one question well — what happened to this item, under this clause, by this model version — and it declines the rest.
Frequently Asked Questions
What does a content-moderation audit-evidence pack contain section by section, and what does each section mean in practice?
What belongs in an evidence pack differs sharply from what platform operators typically preserve during routine moderation. In practice, Content Moderation Audit Evidence reduces to this. For Content Moderation AuditEvidence Pack specifically, five sections: policy-to-prompt mapping (which clause is encoded by which prompt or threshold), per-decision records (what happened to each item), version pinning (which model, prompt, policy and guidance revisions were live), escalation and appeal history, and sampling evidence for actions taken without human review. In practice each section corresponds to a data source with a named owner, so an inquiry resolves by query rather than by reconstruction., model version, prompt revision, policy revision, reviewer-guidance revision, and the threshold in force. Each describes a system state that no longer exists once the pipeline moves on — a retrained classifier cannot report what its predecessor scored. Score, action, and timestamp are usually already logged; the pinning fields are the ones teams discover missing during an inquiry.
How is the policy-to-prompt mapping represented as a section a reviewer can read without engineering help?
As a table with one row per policy clause: clause identifier, published clause text, the encoding prompt or rule, the threshold, and a revision date on each side. Dual dating is what lets a reader in November see which pairing was live in March, since policy wording and operational encoding rarely change on the same day.
What does the per-decision section look like for an automated-only action versus a human-reviewed action?
A human-reviewed record carries the adjudication trail: queue, operator identifier, guidance revision, decision, and time taken. An automated-only record has no adjudicator and should not imply one — it carries the score, threshold, pinned versions, and a pointer into the sampling section, where documented sample design and re-check error rates stand in for per-item human judgement.
How are model and prompt versions pinned to individual decisions inside the pack?
By writing an immutable identifier onto the decision record at the moment the decision is made — a model registry version, a container image digest, or a git SHA for prompt files. The mechanism matters less than immutability: a movable tag is not a pin, because it cannot prove what was running on the decision date.
Which sections stay fixed and which are populated per inquiry, so the pack survives policy revisions?
The five-section anatomy is invariant. Contents are drawn per inquiry, filtered to the decisions, dates and clauses in question. A policy revision adds rows to the mapping section and new revision identifiers to the pinning section without changing the structure, which is why the same pack serves repeat inquiries without a rewrite.
How do the pack sections stay within operational-workflow scope and avoid drifting into speech adjudication or behaviour-tracking claims?
Every section documents how a written policy was applied to specific content by a specific system version. None assesses whether the policy is correct, and none aggregates a user’s activity beyond the items acted on. A section that starts profiling behaviour across a user’s history has left enforcement evidence and entered a different regime with different obligations.s.
Documentation that survives regulatory scrutiny
Auditors will ask for timestamped records of every policy version, every training session, and every appeal—structure your evidence pack accordingly. That answer is workload-specific, and it is worth writing down before you build.
