A reviewer rarely asks how accurate your classifier is. They pick one flagged item out of the log and ask what happened to it — which policy clause was in force, which model version scored it, who adjudicated it, what the user was told. Aggregate evidence cannot answer that question, and the gap is usually discovered mid-inquiry rather than before one.
This is an inventory. Not the anatomy of an evidence pack, not the argument for why accuracy metrics fail as a defence — just the field list for a single decision record, plus the awkward parts of it that trust teams tend to leave for later. It is scoped to operational moderation workflow: content flagged, actioned, appealed. It does not extend to political-speech adjudication, and it does not cover behaviour-tracking of users across a platform.
What “per-decision evidence” means in practice
The practical definition is a retrieval test, not a documentation test. Per-decision evidence exists when a single content ID resolves to a complete, timestamped, immutable decision record without an engineer reconstructing it from logs. If reconstruction is required, you have logs; you do not have evidence.
That distinction has teeth because reconstruction degrades over time in ways a document does not. Model weights get retrained. Prompt text gets edited in place. Reviewer guidance gets a new revision without a version number. Log retention windows expire at 30 or 90 days while an inquiry arrives at month seven. Each of those is survivable individually. Together they mean the answer to “what happened to this item” is assembled from partial traces and hedged with “as far as we can tell” — which is precisely the sentence a reviewer records as a finding.
We see the same sequence in most trust-and-safety engagements: the aggregate reporting is in decent shape, the per-item retrieval path was never designed, and the first sample request turns into a two-week engineering project. Nothing in that pattern is unusual, and nothing about it is a modelling problem.
The record inventory
Seven field groups make up a defensible record. The distinction that matters is between fields that are non-negotiable — a reviewer treats their absence as a control failure — and fields that strengthen the record but can be reasonably deferred.
| Field group | What it holds | Status |
|---|---|---|
| Content identity | Stable content ID, capture timestamp, content hash or decoded-frame reference, surface (feed, comment, upload) | Non-negotiable |
| Policy in force | Policy clause identifier plus the clause revision that was live at decision time — not the current wording | Non-negotiable |
| Evaluating system | Model or prompt version identifier, threshold applied, score returned, ruleset revision | Non-negotiable |
| Human adjudication | Reviewer or queue identity, guidance revision they worked from, decision, decision timestamp, escalation hops | Non-negotiable |
| User-facing outcome | Action taken, notice text sent to the user, notice timestamp, appeal route offered | Non-negotiable |
| Appeal trail | Appeal raised, adjudicator, outcome, reason code, whether the original policy interpretation changed | Non-negotiable where an appeal exists |
| Context and secondary signals | Prior enforcement on the same account, secondary classifier outputs, sampling flag for automated-only actions | Strengthening |
Two entries in that table cause most of the practical work.
Policy in force is not the same as policy text. A record that stores the clause identifier but resolves it against the live policy document will silently show a reviewer the wrong wording once the clause is edited. The clause needs a revision identifier stored on the record itself, and the historical text has to remain retrievable. Editing policy in place is the most common way an otherwise good record set becomes indefensible.
Notice text belongs in the record, not in the mail system. The question “what was this user told” is asked in almost every appeal dispute, and templated notices change. Storing the rendered notice — not the template ID — closes a category of dispute that is really about missing context rather than genuine policy disagreement.
How is a model version pinned to a decision after the fact?
It is not pinned after the fact. That is the point of the question, and the honest answer is that retrospective pinning is a reconstruction, not a record.
Pinning happens at inference time: the serving layer writes the model or prompt identifier into the decision event in the same transaction that writes the score. In practice that means the classifier’s artefact digest — a model registry version, an MLflow run ID, a container image digest, a prompt-template hash — travels with the response rather than being looked up from a deployment table later. Deployment tables are the usual failure: they record which version was live during a window, which breaks the moment two versions ran concurrently during a canary rollout, or a rollback happened mid-day.
Version pinning is engineering discipline before it is governance discipline, and it is the same discipline that reliability work depends on — a moderation record inherits it rather than inventing it.
Retention, immutability, and not hoarding user data
These three pull against each other, which is why they are usually deferred and then discovered together.
Immutability is demonstrated structurally, not asserted. Append-only decision events, write-once storage or an equivalent retention lock, and a per-record integrity hash chained across the event sequence — those are the mechanisms a reviewer recognises. A mutable table with an updated_at column and an audit log maintained by the same service is not an immutability control, because the same credentials can rewrite both.
The tension with data minimisation resolves by separating the decision record from the content. The decision record — clause revision, model version, threshold, score, reviewer, notice — is small, contains little personal data, and can be retained for the full inquiry-relevant window. The content itself, and the personal data attached to it, follows the shorter retention schedule; the record keeps a hash and a reference. A reviewer sampling a decision from eight months ago can then see exactly what governed it, without the platform having retained the underlying media longer than its own policy allows. Where the content genuinely must be reproducible — video especially — the decode path has to be pinned too, not just the file.
The uncomfortable part is that retention windows for decision records are usually set by whichever system happened to store them, not by a decision anyone made. Naming the window is a five-minute exercise that most teams have never done.
Measuring completeness before someone samples it
Record completeness is a percentage, which makes it reportable. Take a defined denominator — all enforcement actions in a period — and measure the share carrying every non-negotiable field, broken out by field so gaps are attributable rather than aggregate.
That number is what turns a reviewer’s sample request into a query rather than a project. A trust team that already reports 97% completeness with a named gap in appeal reason codes is in a different conversation from a team that has to go and find out. The operational payoff is the same either way: single-decision reconstruction drops from days of log archaeology to minutes, and a 50-item sample becomes a same-week response.
Completeness is also the metric that reveals design defects early. A persistent gap in “guidance revision the reviewer worked from” almost always means reviewer guidance is not versioned at all — a policy-operations problem surfaced by an evidence metric.
Where per-decision evidence stops
Per-decision records answer “why this item”. They do not answer “is the system working”, and reviewers ask both. Systemic-risk reporting, category-level precision and recall, appeal-overturn trends, sampling rates for automated-only actions — those are aggregate surfaces built over the per-decision records, and they are a separate deliverable with separate consumers. The reason to keep the boundary sharp is that aggregate reporting derived from incomplete per-decision records inherits the incompleteness silently. Our broader treatment of how these evidence layers assemble into a defensible governance position sits in the AI governance and trust practice, where the record layer described here is the section a reviewer actually samples from.
The open question we have not seen answered well anywhere: how much of the per-decision record should be visible to the user whose content it concerns, rather than only to a reviewer? The evidence exists either way. Who gets to read it is still being decided by policy rather than by engineering.
Frequently Asked Questions
What does per-decision evidence an external reviewer expects to see mean in practice?
Per Decision Moderation Evidence rarely needs a long answer. For Per Decision Moderation Evidence, it helps to be precise. In practice, Per Decision Moderation Evidence reduces to this. It means one content ID resolves to one complete, timestamped, immutable record covering policy clause revision, evaluating model version, threshold and score, reviewer adjudication, user notice, and appeal outcome. The practical test is retrieval time: if answering takes an engineer reconstructing from logs, the evidence does not exist yet.
What fields make up a complete per-decision moderation record, and which of them are non-negotiable?
Content identity, policy clause revision in force, evaluating model or prompt version with threshold and score, human adjudication details, and the user-facing outcome including notice text are all non-negotiable; the appeal trail is non-negotiable wherever an appeal exists. Prior-enforcement context and secondary classifier signals strengthen the record but can be deferred.
How is the model or prompt version that made a given decision pinned to that decision after the fact?
It cannot be, reliably — retrospective pinning from a deployment table breaks during canary rollouts and mid-day rollbacks. The version identifier must be written into the decision event at inference time, travelling with the response as an artefact digest such as a registry version, run ID, or prompt-template hash.
What does the reviewer adjudication and escalation trail need to show for a single item, including appeal outcomes?
Reviewer or queue identity, the guidance revision they worked from, the decision and its timestamp, and each escalation hop. On appeal it must additionally show who adjudicated, the outcome, a reason code, and whether the original policy interpretation changed rather than only whether the action was reversed.
How long must per-decision records be retained, and how is immutability demonstrated without retaining more user data than necessary?
Separate the decision record from the content: the record is small, carries little personal data, and can be retained across the full inquiry-relevant window, while the content follows its own shorter schedule and is referenced by hash. Immutability is demonstrated with append-only events, write-once or retention-locked storage, and chained integrity hashes — not with an audit log written by the same service that can modify the record.
How does a trust team measure and report per-decision record completeness before a reviewer samples it?
Define the denominator as all enforcement actions in a period, then report the share carrying every non-negotiable field, broken out per field so gaps are attributable. Field-level gaps are diagnostic — a persistent gap in reviewer guidance revision usually means guidance is not versioned at all.
Where does per-decision evidence stop and aggregate or systemic-risk reporting begin?
Per-decision records answer “why this item”; aggregate reporting answers “is the system working” and is built over those records. Keeping the boundary sharp matters because aggregate figures derived from incomplete per-decision records inherit that incompleteness without showing it.
Building an evidence bundle external counsel can defend
Attach screenshots, classifier outputs, policy version, and moderator notes to every decision—external reviewers cannot reconstruct context from database IDs alone.
