Sign-off on a monitoring harness is usually treated as one approval at handoff: the project sponsor looks at a complete-looking deliverable and accepts it. That works until someone asks who approved the drift baselines. A monitoring harness has no single competent signer — each section asserts a different kind of claim, and the roles that can defensibly attest to those claims are different people.
This is not a bureaucratic point. It decides whether a release-readiness review closes in one cycle or three, and whether the question “who approved this model release?” is answerable in minutes during an incident.
What “who signs each section” means in practice
A signature on a harness section is an attestation with a specific scope. An engineering lead signing a regression suite is asserting that the suite runs, is deterministic under a fixed seed, and fails when it should. That is a real and checkable claim. It is not the same claim as “these pass criteria match what the business agreed to accept” — which the engineer who wrote the suite is structurally the wrong person to make, because they chose the criteria.
So the per-section signer map answers two questions per section, not one:
- Which role has the competence to verify this section?
- What exactly does their signature assert — and what does it deliberately not cover?
The second question is the one teams skip. A signature with undefined scope is worse than no signature, because it creates the appearance of review where none of substance happened. We see this most often on drift telemetry: a section accepted by someone who cannot say what the baseline distribution was, over which window it was computed, or what a breach would trigger.
The signer map
The mapping below is the reviewer column of the harness deliverable. Sections come from the standard harness shape; the roles are the ones we ask clients to name and staff at scoping, before any instrumentation work begins.
| Harness section | Primary signer | What the signature asserts | Explicitly out of scope |
|---|---|---|---|
| Eval harness | Engineering lead | The suite executes reproducibly against a versioned dataset; results are traceable to a run ID | Whether the metric is the right business metric |
| Eval dataset & slices | QA / validation owner | Slices reflect the acceptance spec and the operational domain; no leakage from training data | Suite implementation quality |
| Regression suite (mechanics) | Engineering lead | Fixed-seed determinism; known-failure cases pinned; CI wiring works | Whether the pass bands are acceptable |
| Regression pass criteria | QA / validation owner | Thresholds and behaviour bands match the agreed acceptance criteria | How the thresholds are computed |
| Drift telemetry & baselines | Data science owner (engineering side) | Baseline windows, features monitored, and statistical method are documented and reproducible | Whether an alert is actionable at the client’s staffing level |
| Alert thresholds & escalation | Customer operations owner | Alerts are actionable by the on-call rota that actually exists; escalation path names real people | Correctness of the underlying drift computation |
| Release-readiness review record | Named release approver (sponsor or change board) | All section signatures present and in date; residual risks accepted | Any section-level technical claim |
| Audit-evidence pack | Quality function (QA in regulated settings) | Evidence is retrievable, attributable, and matches the signed sections | Engineering fitness of the artefacts themselves |
Two things about this table matter more than the specific role names, which vary by organisation. First, the mechanics of a section and the acceptance criteria for that section are split across two signers. Second, alert thresholds are the one section the vendor cannot sign at all — actionability is a property of the customer’s staffing, not of the model.
Where QA signs instead of engineering
The regression suite is where this boundary gets tested hardest, because it looks like one artefact. It is two. The suite is code — engineering territory. The pass criteria encoded in that code are a specification of acceptable behaviour — QA territory.
Collapsing them produces a predictable failure: the suite passes on every run because the bands were set wide enough to accommodate the model that already existed. Nobody lied. The person who chose the bands also chose the implementation, and the implementation informed the bands. Splitting the signature breaks that loop. We treat a regression section with a single signer as an unsigned section in the reliability scorecard, and it downgrades the dimension accordingly.
The same split applies to the eval harness: engineering attests to reproducibility, QA attests to whether the slices cover the operational domain. A perfectly reproducible eval over the wrong slices is a well-engineered irrelevance. The production AI reliability practice is where we set out how these harness sections are built and staffed as a single deliverable rather than a checklist.
Regulated settings: where the chain gets longer
Under GxP and GAMP-aligned quality systems, the signing chain acquires a role the general case does not have: an independent quality function that signs the evidence pack without signing any technical section. Its attestation is about traceability and attributability — that each signature is from a trained, authorised person with the competence recorded, and that the artefacts behind each section are retrievable in their signed state.
That role cannot be filled by the engineering team, and it cannot be added retrospectively. A pack assembled after the fact, with attestation collected by email months later, is what an auditor calls reconstructed evidence. It is admissible only under argument, which is precisely the position a reliability engagement exists to avoid. In life-sciences and automotive contexts we scope the quality signer into the reviewer matrix at the start, with the harness sections mapped to their validation-plan equivalents.
Retrain and re-sign: delta, not full
Signatures decay when the thing they attest to changes. The naive responses are both wrong: treat the signatures as permanent (they no longer describe the deployed system), or re-sign everything on every retrain (nobody sustains that, so it degrades into rubber-stamping within two cycles).
The workable rule is a delta re-sign keyed to what the update touched:
- Model weights only, harness unchanged — engineering re-signs the regression and eval runs against the new build; QA’s criteria signature carries forward; drift baselines are re-signed only if the baseline window has been recomputed.
- Threshold or slice change — QA re-signs the affected criteria; the operations owner re-signs any alert threshold derived from them.
- New feature, new input source, or new deployment context — full re-sign, because the operational domain the slices were meant to cover has changed.
The release approver’s signature is never carried forward. It is per release by construction, and it asserts only that the section signatures beneath it are present and current. How the harness itself is versioned across those updates is a separate discipline from who signs it; the mechanics of that update path are covered in how a monitoring harness gets updated when the model updates.
Writing it into the statement of work
The reviewer matrix belongs in the SoW, not in the handoff email. Three lines are enough to change the outcome: the section list, the named role against each section (title, not person, so holidays do not stall a release), and a stated turnaround for review. Buyers who fix this at scoping typically close release-readiness review in one cycle rather than three — an observed pattern across our reliability engagements, not a benchmarked figure, and it depends heavily on whether the customer can actually commit an operations owner.
It is also a useful question to put to competing vendors during selection. Ask which harness sections they will sign, and which they expect you to sign. A vendor who offers to sign the alert thresholds has either misunderstood the deliverable or is willing to attest to something they cannot know.
Frequently Asked Questions
What does ‘who signs each section of a production AI monitoring harness’ mean in practice?
Each section of a production AI monitoring harness requires sign-off from specific stakeholders with domain authority over that component. It means each harness section — eval harness, regression suite, drift telemetry, alert thresholds, release-readiness review, audit-evidence pack — carries a named reviewer role and a defined attestation scope, agreed before the work starts. In practice it is a table in the statement of work with three columns: section, signing role, and what that signature asserts.
Which harness sections belong to engineering, and what exactly does an engineering signature assert?
Engineering owns the mechanics: the eval harness executes reproducibly against a versioned dataset, the regression suite is deterministic under a fixed seed and fails when it should, and drift baselines are computed by a documented method over a stated window. The signature asserts reproducibility and traceability to a run ID — not that the metrics chosen are the right ones commercially.
Where does QA sign rather than engineering, and why does that boundary matter for regression suites and acceptance criteria?
QA signs the acceptance criteria and the slice coverage; engineering signs the implementation. The boundary matters because whoever writes a regression suite also implicitly chooses its pass bands, and bands chosen alongside the implementation tend to accommodate the model that already exists. Splitting the signature breaks that loop.
What does the customer or operations owner sign for — alert thresholds, escalation paths, or model behaviour?
Alert thresholds and escalation paths, not model behaviour. Only the operations owner can attest that an alert is actionable at the staffing level they actually run, and that the escalation path names real people with real on-call coverage. Vendors cannot sign this section, because actionability is a property of the customer’s organisation.
When does a regulator or a regulated-industry quality function need to be in the signing chain, and how does that map to GxP/GAMP roles?
In GxP and GAMP-aligned environments an independent quality function signs the audit-evidence pack without signing any technical section — its attestation covers traceability, attributability, and that each signer was trained and authorised. It maps to the quality-unit review role over a validation plan, and it must be scoped in from the start; retrospective attestation reads as reconstructed evidence.
What happens to existing signatures when the model is retrained or the harness is updated — full re-sign, or delta re-sign?
Delta re-sign, keyed to what changed. Weights-only retrains need engineering to re-sign the eval and regression runs while QA’s criteria signature carries forward; threshold or slice changes pull QA and the operations owner back in; a new feature or deployment context triggers a full re-sign. The release approver’s signature never carries forward.
How should a buyer write the reviewer matrix into a statement of work so sign-off does not stall at handoff?
List the harness sections, name a signing role (job title, not individual) against each, and state a review turnaround per section. Naming roles rather than people prevents a single absence from stalling a release, and agreeing it at scoping surfaces which internal roles the buyer needs to commit before the engagement starts.
Signature authority mirrors incident accountability
Every section of your monitoring harness should map to exactly one person who will be woken at 3 a.m. when it fires — ambiguity here becomes operational debt within weeks. Everything else is detail.
