A model accuracy figure and a screenshot of the review tool are not evidence. When a platform-trust reviewer asks how a model-assisted moderation decision was made, the question underneath is reconstructability: can someone replay how this specific item reached this specific outcome, using records that existed at the time rather than a narrative assembled afterwards? An evidence pack that answers that question defends the workflow. A pack that reports aggregate model metrics leaves the adjudication step undocumented, which is exactly the step the reviewer cares about.
A moderation evidence pack is a standing deliverable with four artefact groups: the triage threshold policy in force, the queue telemetry, the reviewer adjudication record, and the version history that binds each decision to a model build and a policy revision. Miss any one of the four and single-decision replay breaks.
What does an operational content-moderation evidence pack contain in practice?
The pack is an inventory, not a report. Each artefact has an owner, a retention window, and a defined role in reconstruction.
| Artefact | What it records | Produced by | Role in replay |
|---|---|---|---|
| Triage threshold policy (versioned) | Confidence bands per category, severity mapping, auto-action boundaries, abstain default | Engineering, signed off by policy owners | Explains why the item was routed where it was |
| Queue telemetry | Arrival time, severity band, queue depth at arrival, routing decision, wait time, reviewer touch count | Engineering | Shows what the workflow actually did, not what it was designed to do |
| Reviewer adjudication record | Reviewer role/specialisation, decision, timestamp, escalation path taken, policy clause cited | Trust and safety operations | Documents the human decision the model did not make |
| Model and policy version history | Model build ID, training/eval snapshot reference, threshold revision, policy revision, effective date ranges | Engineering (model), policy owners (policy) | Binds the decision to the exact configuration in force |
| Metric definitions | How each operating metric is computed, at what granularity, over what window | Engineering | Makes the numbers comparable across reporting periods |
| Sampled decision walkthroughs | 10–20 end-to-end reconstructions across severity bands, including at least one abstain and one overturn | Engineering + operations jointly | Demonstrates the pack works before anyone tests it under pressure |
That last row is the one teams skip and then regret. A pack that has never been used to reconstruct a decision is an untested backup.
Which artefacts engineering owns, and which it must not
The split matters because it is where scope disputes start. Engineering owns the evidence layer: the telemetry, the version history, the metric definitions, the threshold mechanics. Policy owners own the categories, the severity classification, and every adjudication of substance. We supply the record that shows a decision was made under a stated configuration; we do not supply the judgement that the decision was correct.
In practice the adjudication record is the boundary artefact — engineering builds the field schema and enforces that it is populated, while the content of each field belongs to the reviewer. If an engineering team finds itself drafting the policy clause list that reviewers cite, the ownership line has already moved. We treat that as a scoping defect rather than a documentation gap, and the reasoning behind the carveout is developed further in our note on where operational moderation ends and policy adjudication begins.
What has to be logged at decision time
Reconstruction cannot be retrofitted. The pack is only assemblable if the logging was correct when the item passed through, which makes this a design constraint on the pipeline rather than a reporting task.
The minimum decision-time record for a single item:
- Content item identifier and the hash or fingerprint used for matching, where matching applies
- Model build ID and the raw score per category — not just the thresholded band
- Threshold revision ID in force at that timestamp
- Routing outcome and the rule that produced it, including abstain events
- Queue state at arrival: depth in the target severity band, current staffing level
- Every reviewer touch, in order, with role and outcome
- Policy revision ID in force at the adjudication timestamp
Storing the raw score alongside the band is the detail with the highest payoff. Thresholds get retuned; a band recorded without its underlying score becomes uninterpretable the moment the policy changes, whereas a stored score can be re-evaluated against any later revision. This is standard practice in ML observability stacks — the same instinct behind logging model inputs and outputs through MLflow or an equivalent registry rather than logging only the final label — and it applies with more force here because the audit horizon is legal rather than diagnostic.
The metrics the pack defines
A pack carries metric definitions, not just metric values, because the numbers only mean something if their computation is fixed. Four operating metrics belong in every moderation pack:
- Queue depth by severity band, before and after triage — measured at fixed intervals, not as a daily average, since averaging hides the peaks that cause service-target breaches.
- Time-to-first-review on high-severity items — from arrival, not from routing, and reported as a distribution with a stated percentile rather than a mean.
- False-positive review load — the share of reviewer time spent on items the model escalated and the reviewer cleared, which is the honest cost of a conservative threshold.
- Model–reviewer agreement drift per period — agreement rate between the model’s band and the reviewer’s outcome, tracked as a trend, with the sampling method stated.
Instrumentation for these metrics is a separate engineering job with its own failure modes; we work through the queueing-system view of it in how to instrument a moderation workflow for review latency and accuracy. The pack consumes that instrumentation and pins the definitions so that a figure quoted in one review round means the same thing in the next.
Refresh cadence and what forces an out-of-cycle rebuild
Standing beats on-demand. Teams that assemble these artefacts on a schedule avoid the reconstruction scramble that dominates unplanned evidence requests — the observed pattern in engagements we have worked on is that the reconstruction work, not the analysis, consumes most of an unplanned cycle (an engagement observation, not a benchmarked figure). A quarterly refresh with a rolling sample suits most platforms. Four events should force an out-of-cycle rebuild: a model version promotion, a threshold revision, a policy revision that changes category definitions, and any change to reviewer routing or specialisation.
What the pack deliberately excludes is as load-bearing as what it includes. No user-behaviour profiles, no cross-session risk scores, no reviewer performance rankings, and no defence of the policy itself. The unit of evidence is the content item plus the workflow that handled it. Widening that unit turns a workflow record into a surveillance record, which is a different system with different obligations — a boundary we treat as fixed rather than negotiable.
The pack sits inside a broader moderation programme where triage, escalation design and reviewer capacity all interact; our work on model-assisted moderation for platform and broadcast operators is described under media and telecom engineering, and the validation-pack deliverable itself is scoped through our services engagements.
So the question worth asking before the next evidence request arrives is not whether the pack exists. It is whether anyone has tried to replay a decision from six months ago using only what was logged at the time — and what broke when they did.
Frequently Asked Questions
What does an operational content-moderation evidence pack contain in practice?
Four artefact groups: the versioned triage threshold policy in force, queue telemetry covering arrival through routing and wait times, the reviewer adjudication record for each human decision, and the version history binding each decision to a model build and policy revision. Alongside those sit the metric definitions and a set of sampled end-to-end decision walkthroughs that prove the pack can actually be used for replay.
Which artefacts in the pack are produced by engineering, and which must come from the policy or trust and safety side?
Engineering produces the telemetry, version history, metric definitions and threshold mechanics — the evidence layer. Policy and trust and safety own the category definitions, severity classification and the substance of every adjudication. The adjudication record is the boundary artefact: engineering defines and enforces the schema, reviewers own the content of each field.
How is a single model-assisted moderation decision reconstructed from the pack — what has to be logged at decision time?
Replay requires the item identifier, the model build ID with raw per-category scores, the threshold revision in force at that timestamp, the routing outcome and the rule that produced it, queue state at arrival, every reviewer touch in order, and the policy revision at adjudication time. Storing raw scores rather than only the thresholded band is the detail that keeps old decisions interpretable after thresholds are retuned.
How are triage thresholds, model versions and policy revisions recorded so the pack stays valid as any of them change?
Each is a versioned object with an effective date range, and every logged decision carries the specific revision IDs in force at its timestamp rather than a pointer to the current configuration. Model promotions, threshold revisions, category-definition changes and routing changes each trigger an out-of-cycle rebuild of the pack.
Which metrics does the pack define, and how are queue depth, time-to-first-review, false-positive review load and agreement drift measured consistently?
The pack pins definitions, not just values: queue depth sampled at fixed intervals by severity band before and after triage; time-to-first-review measured from arrival and reported as a stated percentile; false-positive load as the share of reviewer time on model-escalated items the reviewer cleared; and agreement drift as a trend in model-band versus reviewer-outcome agreement with the sampling method declared.
How often should the pack be refreshed, and what triggers an out-of-cycle refresh?
A quarterly refresh on a rolling sample suits most platforms as a standing cadence. Out-of-cycle rebuilds are triggered by a model version promotion, a threshold revision, a policy revision that alters category definitions, or a change to reviewer routing or specialisation.
What does the pack deliberately not contain, and why does keeping it scoped to workflow evidence matter?
It excludes user-behaviour profiles, cross-session risk scores, reviewer performance rankings and any defence of the policy itself. The unit of evidence is the content item plus the workflow that handled it; widening that unit converts a workflow record into a user-profiling record, which carries different obligations and a different scope of consent.
Building your evidence pack: three core components
Every operational content moderation evidence pack requires baseline metrics, documented policy decisions, and audit trails that survive leadership turnover. Everything else is detail.
