Signing off an LLM default-model swap and signing off a GPU-serving stack milestone are two different decisions, and one production readiness review checklist serves neither of them well. A model swap is a quality decision. It passes or fails on a versioned scorecard of numeric thresholds (task accuracy, faithfulness, refusal correctness, p95 latency, cost) that is set before any candidate is evaluated. A serving milestone is an infrastructure acceptance decision. It passes or fails on an agreed, representative load profile and an agreed failure-injection scope, written into the acceptance terms before the test runs. Each needs its own document, its own approvers and its own pass criteria. Merge them and you get a sign-off meeting that argues about tokens per second when the real question was whether the new model invents refund policies.

Why can one production readiness checklist not cover both a model swap and a GPU-serving milestone?

A model swap and a serving milestone answer different questions, so they need different evidence. The swap asks whether a different model behaves well enough on your tasks. The milestone asks whether a stack someone built for you holds its latency targets and survives faults under your traffic.

TechnoLynx handles both. TechnoLynx first frames each set of criteria against your contract and risk constraints. TechnoLynx then defines the thresholds and the load and failure profiles on your target serving stack, runs the acceptance harness and produces the sign-off evidence. For the model-choice side, the work starts with the LLM Selection Pack.

The two decisions side by side:

	LLM default-model swap	GPU-serving stack milestone
Decision	Replace the routed default model	Accept, and pay for, a delivered serving stack
Governing document	Versioned scorecard: rubric, locked dataset, evaluators, numeric thresholds	Acceptance criteria agreed in the statement of work or purchase order
Primary evidence	Per-dimension scores and regression flips on a production-sampled dataset, plus p95 latency under realistic concurrency	p99 latency and throughput under the agreed load profile, plus behaviour under injected faults
What a fail triggers	The candidate goes to a fallback role or is rolled back	A written rejection notice describing the tests, the results and each issue, then remediation and a retest
Reversibility	High: a feature flag or routing change can revert it without a deploy	Low: acceptance and payment are contractual events

The model-swap scorecard: numbers fixed before any candidate runs

Treat the swap as a deployment event that passes through the same gates as a prompt or rubric change. The scorecard comes in four parts: a written rubric with numeric thresholds, a locked dataset of 150 to 1,000 production examples split into common, ambiguous and adversarial slices, versioned evaluators calibrated against human labels, and per-dimension operational floors. All four are fixed before candidates are evaluated, so the result cannot shape the decision. A candidate that beats the incumbent on aggregate but regresses on the adversarial slice does not qualify as the default.

What hallucination rate, p95 latency and refusal rate should trigger an automatic no-go?

No published number is correct for every product. What exists are reference points you can anchor to and then adjust per query category and risk tier. Openlayer (2026) suggests flagging when hallucination exceeds 2-3% of scored traffic for customer-facing systems in regulated domains, and 5-8% for lower-consequence internal tools. It also recommends setting the threshold per query category, not uniformly. For grounded, retrieval-based answers, the same vendor’s hard gate example is to block deployment when groundedness falls below 85%. For latency and task accuracy, Scorable gives example floors of at least 0.92 accuracy on the common slice and p95 latency under 2 seconds. Measure that latency under realistic concurrency, because some models degrade non-linearly at scale.

Refusal rate cannot be set separately from hallucination rate. In the HalluLens benchmark (2025), hallucination rate is the share of wrong answers among the questions a model actually attempts, and false refusal rate is tracked alongside it so a model cannot look accurate simply by declining to answer. The spread in refusal rates between models is large. On HalluLens’s PreciseWikiQA task, Llama-3.1-8B-Instruct refused 83.09% of questions while GPT-4o refused 4.13%. The Llama model hallucinated less when it did answer (48.37%, against 85.22% for Qwen2.5 7B), but its high refusal rate pulled down its correct-answer score.

A no-go rule should therefore pair the two metrics. That means a ceiling on hallucination-when-attempted, a ceiling on false refusal on answerable questions, and a negative-flip ceiling on cases the incumbent got right. Scorable’s continuous-monitoring example uses a negative flip rate above 3% on the common slice to trigger recalibration, and faithfulness below 0.90 to trigger a rollback.

Which published LLM benchmarks or harnesses make swap thresholds defensible?

A board trusts a threshold when the metric definition comes from a published benchmark and the number comes from your own data, with error bars. Several published sources cover the metric definitions:

• HalluLens (2025) defines hallucination, false refusal and correct-answer rates, and generates test questions dynamically to limit leakage into training data.
• Vectara’s hallucination leaderboard scores summarisation factual consistency with its HHEM-2.3 model and reports an answer rate next to the hallucination rate.
• OR-Bench holds 80,000 over-refusal prompts across 10 common rejection categories.
• HELM (Stanford CRFM) evaluated 30 models on shared scenarios across seven metrics, including calibration, robustness and toxicity.
• EleutherAI’s lm-evaluation-harness runs evaluations across backends including Megatron-LM, SGLang and ONNX Runtime GenAI.

None of these is your production traffic. Vectara itself advises running its hallucination metric alongside other evaluations, not as a standalone measure.

The statistics are what turn a threshold from an opinion into evidence. Anthropic’s evaluation-statistics paper (2024) recommends reporting standard errors, comparing two models on question-level paired differences, and using power analysis to check whether an eval can detect the effect you care about. In its worked example, with 198 questions, raising resamples per question from 1 to 10 lowered the minimum detectable effect from 13.2% to 7.5%. Without that kind of analysis, a dataset can be too small to separate a real regression from noise. The signoff tool applies this kind of statistical test to model swaps. It runs the prompt suite N times on each model, reports per-case flip rates, and fails the swap only when the new model is less accurate and an exact sign test gives p ≤ 0.05.

How tight can thresholds be without blocking a contractual go-live date?

Set them as tight as customer harm requires and no tighter, and let a staged rollout carry the remaining risk instead of the gate. The Google SRE Workbook warns that an overly strict definition of acceptable behaviour produces false positives, while one that is too loose lets bad releases through. Its error-budget policy lets releases proceed while the service meets its SLO, and halts all but P0 changes once the four-week error budget is spent.

Three choices keep a go-live date safe without weakening protection. First, split thresholds into absolute and incremental ones. Google’s ML Test Score gives the examples “error for slice x must be <5%” and “change in error for slice x must be <1%” against the previous model. Second, make only the customer-harming dimensions hard gates and send the rest to review. Third, ship behind a canary. In the Workbook’s arithmetic, a release that fails 20% of requests, exposed to a 5% canary, costs a 1% overall error rate. Cheaper models that miss the default’s thresholds can still serve as fallbacks for cost-sensitive batch work, with their lower thresholds documented.

Signing off a GPU-serving stack milestone

A serving milestone is accepted when the delivered stack passes the acceptance tests agreed in the purchase order or statement of work. The number matters less than the conditions under which it was produced, which is why both sides fix those conditions in writing first. If the milestone is part of wider serving work, TechnoLynx’s GPU performance engineering practice covers the tuning that comes before acceptance.

What load profile must a p99 latency and throughput test replicate to count as representative?

A representative profile reproduces your request mix, arrival pattern, concurrency and hardware, and it excludes warmup. The checklist below is what we would expect to see in the acceptance schedule:

1. Request mix. Input and output length combinations per use case, because benchmark sweeps run across use cases and concurrency values together.
2. Arrival process. MLPerf’s server scenario uses Poisson arrivals at a target query rate. vLLM’s bench serve adds a burstiness parameter: values below 1 make arrivals burstier through a gamma distribution, values above 1 make them more uniform. BurstGPT’s Azure traces show that conversation services follow periodic patterns while API services arrive in aperiodic bursts. In BurstGPT’s evaluation, that burstiness caused KV cache bottlenecks and spikes in failure rate.
3. Concurrency. A --max-concurrency cap reproduces an upstream component that limits in-flight requests, separately from the request rate. GenAI-Perf sweeps concurrency so you can read off the highest throughput that stays within a time-to-first-token budget.
4. Latency budget per token, not only per request. MLPerf’s Llama 2 70B benchmark caps time per output token at 200 ms, which it relates to human reading speed.
5. Recorded traffic over synthetic traffic. In the Vanguard study (2026), calibrating load tests with recorded traffic reduced capacity-estimation error from about 30% to 2-6%.
6. Warmup exclusion. JIT compilation, CUDA kernel caching and weight loading make the first 2-5 minutes unrepresentative.
7. Target hardware. Vanguard found a 4.9% systematic bias between GPU generations. Its preliminary experiments with generative models, which were outside the four model classes it studied, showed higher measurement variance.
8. Measurement window. Make the window long enough for many requests to finish. NVIDIA suggests around 100 seconds for larger models at high concurrency.

Synthetic load maximises code coverage but gives poor state coverage in systems with caches and request affinity. An LLM server with a KV cache and prefix caching is exactly that kind of system. Choosing the hardware is a separate question from accepting the stack. LynxBenchAI informs that choice by measuring throughput, latency, memory, energy and cost per unit of work across accelerators. It does not replace an acceptance test on your own stack.

Which failure-injection scenarios (GPU node loss, network partition, OOM) are mandatory sign-off evidence?

We recommend treating every fault the architecture claims to tolerate as mandatory, and compound or exotic faults as optional unless the contract promises them. The 2026 study “Beyond Microservices”, which tested a Ray and Kubernetes LLM stack, injected four faults one at a time: CPU hog, an 8GB memory leak, one second of network latency, and GPU throttling from 300W to 150W. Its load was a constant five queries per second of chat traffic, averaging 512 prompt tokens and 256 output tokens. Each fault type was injected 30 times. NVIDIA’s Dynamo fault-tolerance framework adds GPU XID errors, memory pressure, and partitions between frontend and workers or between workers. It runs tests through standard, overflow and recovery phases.

Two findings from that study shape the scope. Memory faults killed Ray actors through OOM and made replicas temporarily unavailable. GPU throttling slowed token generation while standard GPU utilisation and memory metrics stayed unchanged. In that study, TTFT and token rates showed the throttling fault where standard GPU utilisation and memory metrics did not.

Scenario	Our recommended status	Pass evidence we would propose
Worker or GPU node loss mid-generation	Mandatory where redundancy is claimed	In-flight requests complete or fail cleanly; p99 during recovery within the agreed bound
GPU OOM / memory pressure	Mandatory	Replica restarts, service stays partially available, recovery time recorded
Network partition, frontend to workers	Mandatory	Routing excludes the cut-off workers; error rate stays within the agreed ceiling
GPU thermal or power throttling	Mandatory for latency SLOs	Latency degradation detected and alerted
Specific XID codes, etcd HA, concurrent faults	Optional unless contracted	Run if the architecture depends on them

Run these tests only in a dedicated test cluster. NVIDIA warns that its injection service can cause arbitrary pod kills, partitions and GPU XID errors across the cluster.

Should the client agree in writing on the test harness and methodology before the acceptance test runs?

Yes. Without that agreement, a disputed result has no reference to settle it. Acceptance clauses already have the right shape. Law Insider’s samples include a supplier’s standard procedure plus mutually agreed tests named in the purchase order or statement of work, a fixed testing period (seven days in one sample), and a written rejection that describes the tests, the results and each issue. Some clauses also state that minor failures which do not materially affect operational use cannot fail acceptance. We would add the harness version, the load profile and warmup rule, the hardware, the fault list and the retest procedure to that written agreement.

Who signs off a default model swap, and how fast must it revert?

A model swap needs a documented process with four parts: thresholds set in advance, named approvers for each risk tier, an evidence package tied to a version ID, and rollback criteria that have been tested. With those in place, approval does not depend on a late-night call. TechnoLynx helps set the thresholds for the model’s risk tier, designs the content of the evidence package, and builds the regression harness and staged-rollout mechanics where the swap runs on accelerated compute.

Named approvers and the evidence package

The evidence package records what was tested, which thresholds applied, what passed, what failed and who signed off. Openlayer (2026) treats evaluation logs without timestamps or reviewer sign-offs as incomplete. How many approvers a swap needs depends on its risk tier. The US banking agencies’ SR 26-2 (April 2026) asks for testing rigour commensurate with model complexity and materiality. In Databricks’s example policy, tier-1 models need approval from an independent validator group, tier-2 models can be promoted by a team lead plus a validator, and tier-3 model owners can promote within their own workspace. For a customer-facing LLM route, we would expect engineering to own the scorecard result. Legal or risk would cover regulated query categories, and customer success would cover the support impact. The tier decides which of these approvals are required and which only need to be informed. Separating who builds a change from who approves it is a governance design question in its own right, beyond the scope of this checklist.

Rollback criteria measured in minutes

Fast reverts come from design, not from heroics. DeepInspect’s gateway guidance keeps three routing positions (previous, current and candidate), shifts traffic in stages, and gives a fast revert to the previous rule. Its targets are under a minute for a behavioural or configuration rollback and under five minutes for a routing rollback, including a check of response shape. The shape check matters because a different provider can return responses downstream consumers do not expect. The canary should roll back automatically when its error rate drifts too far from the control. Test the rollback, don’t just write it down. DeepInspect recommends drilling it monthly or after every significant change. The ML Test Score paper also observes that canarying catches problems long after the decisions that caused them, so it supplements earlier tests and does not replace them.

The same structure for quantised, recommender, backtest and digital-twin sign-offs

The structure carries over to other model changes. For an INT8 replacement of a margin-calculation model, we would set a tail-error bound against the full-precision model, approved by model risk. SR 26-2 names adjustment, recalibration or redevelopment as responses when outcomes analysis finds deviations outside established performance thresholds. For a recommender, we would tie offline metrics, A/B results and a fairness breakdown to one version ID. Openlayer suggests flagging any demographic parity gap above 5% for human review. For a credit backtest, Databricks’s reading of the guidance calls for versioned, reproducible challenger and benchmarking runs, not a one-time memo. For a digital twin, the sign-off should formally authorise planners to rely on its predictions once validation passes. Copy the structure, not the numbers. Each domain’s thresholds have to be justified on its own terms, or a committee can reasonably reject the package.

Who can define and run release-readiness sign-off for our LLM swaps and GPU-serving stack?

TechnoLynx defines and runs both kinds of release-readiness sign-off. For a model swap, TechnoLynx agrees the scorecard’s thresholds and dataset before any candidate runs, then produces the paired, error-barred comparison the approvers sign. For a serving milestone, TechnoLynx writes the load profile and failure-injection scope into the acceptance criteria, runs the harness on the target stack, and delivers the evidence against those criteria.

This kind of engagement suits teams without a scorecard, harness or acceptance schedule yet, or teams with a milestone dispute coming. If you already run a versioned evaluation in CI and a load and chaos suite you trust, building in-house is the sensible route. The same applies if sign-off is mainly a legal or regulatory judgement, which needs counsel or an independent validator, not an engineering supplier.