“How much accuracy do you lose if you lower precision?” People ask this expecting a number — some universal percentage they can memorize and apply across models, tasks, and deployment settings. A rule of thumb that makes the trade-off simple. The search for that number is understandable, but the number doesn’t exist. Accuracy loss from reduced precision is not a constant, not even approximately. It depends on what the model is doing, how you measure success, and what kinds of errors your application can tolerate. Two models from the same architecture family, evaluated on different tasks with different metrics, can produce entirely different “accuracy loss” stories from the same precision change. This isn’t a hedge. It’s the structural reality of how numerical representation interacts with task-level evaluation, and skipping it leads to one of two equally bad outcomes: teams avoid precision reduction entirely out of unfounded fear, or they adopt it blindly because it “worked for someone else.” Why sensitivity varies — and why architecture alone won’t predict it Precision changes the numerical regime of execution. Intermediate values get rounded differently, small activations may underflow in FP16, accumulations may lose trailing precision in INT8 paths — the same logic behind quantization as controlled approximation, not model damage. Whether any of that affects the final output depends on what the model is trying to do and where numerical sensitivity actually lives in the computation. Some tasks are naturally robust because their evaluation criteria are coarse relative to the perturbation that format changes introduce. Open-ended text generation, for example, is often evaluated on fluency, coherence, and factual accuracy — dimensions where the difference between BF16 and FP32 intermediate computations rarely produces a distinguishable delta in the final output. The logits shift slightly, but the generated text is effectively the same by any reasonable measure. The BF16 versus FP16 distinction matters here too: BF16 trades mantissa bits for the FP32 exponent range, which is why training pipelines and many inference paths reach for it first — see BF16 vs FP16 dynamic range and precision for the structural reasoning. Other tasks are sensitive in specific ways. Classification on ambiguous inputs, where small changes in logit values cross a decision boundary, can be affected. Regression tasks with tight accuracy requirements on rare edge cases can amplify precision effects that average-case metrics don’t detect. Models with numerically unstable intermediate operations — poorly conditioned normalisation, very deep residual chains, certain loss formulations — can behave differently under reduced precision in ways that are hard to predict without running them. We encounter this asymmetry regularly. A team reduces precision across a set of models, tests headline accuracy, sees no change, and ships with confidence. Later, a subset of users reports degraded behaviour on a rare but important class, and the investigation traces it to a precision-sensitive corner of the model’s input distribution that the headline metric was too coarse to capture. “Accuracy” is not a single metric, and treating it as one hides risk A significant part of the problem is that “accuracy” in practice means whatever metric is easiest to report, and that metric may not be the one that captures the risk you care about. Top-1 classification accuracy on a standard evaluation set tells you about average-case behaviour on that distribution. It says very little about tail behaviour, about calibration, about confidence distribution shifts, or about error characteristics that matter for downstream systems. A precision change can preserve a headline metric while shifting the error distribution in ways that matter operationally — more errors concentrated in a particular class, degraded calibration that makes confidence scores less reliable, or behavioural changes on out-of-distribution inputs that the standard eval set doesn’t contain. This is why a superficial evaluation — “accuracy didn’t change, we’re fine” — can pass at evaluation time and fail in the field. The question isn’t just “did the top-line number move?” It’s “did it move in a way that matters for how this model is actually used?” The point generalises: the decision to run at reduced precision is fundamentally an engineering judgment about controlled approximation, and that judgment is only as good as the evaluation criteria supporting it. Robustness is empirical, not transferable Robustness to precision reduction is not evenly distributed across models, and it’s not reliably predictable from architecture details. Models that look structurally similar — same transformer architecture, same parameter count, similar training recipe — can have different sensitivity profiles because training dynamics, normalisation behaviour, data distribution characteristics, and initialisation randomness all influence where numerical sensitivity ends up living in the model. A model trained with aggressive gradient clipping and stable normalisation might tolerate FP8 inference with minimal quality impact. A model from the same family trained under different conditions might show visible degradation. This makes one common inference pattern particularly unsafe: “we tested reduced precision on Model A and it was fine, so it will be fine on Model B.” That’s not necessarily wrong, but it’s an unvalidated assumption, and unvalidated assumptions about precision behaviour have a track record of eventually producing surprises. The only reliable answer comes from evaluating the specific model-task-metric combination under the precision regime you intend to deploy. How do you assess precision risk for a specific task? The point here is narrower and more actionable than “lower precision hurts accuracy”: accuracy impact is task-dependent, and precision risk assessment must therefore be criteria-driven. Precision risk assessment checklist Step What to do What it produces 1. Define “correct” State what acceptable output quality means for this specific application A written acceptance bar, not a vibe 2. Identify unacceptable errors Name failure modes (class confusion, calibration drift, tail-case degradation) that would be operationally harmful A prioritised risk list 3. Choose evaluation criteria Select metrics that capture those failure modes — not just headline accuracy, but per-class performance, calibration, and tail behaviour A metric suite that can actually fail the model 4. Measure at target precision Evaluate the model under the precision regime you intend to deploy (FP16, BF16, INT8, FP8), on representative data, on the actual runtime (CoreML, ONNX Runtime, TensorRT) Empirical quality numbers for the deployed configuration 5. Decide with evidence Compare the observed quality change against the acceptance criteria from step 1 A go / no-go grounded in the task, not the architecture In practice this means defining what “correct” means for the application, choosing evaluation criteria that reflect that definition, measuring under representative conditions, and deciding whether the observed change is acceptable. Note that step 4 has to happen on the runtime you actually ship on: INT8 on CoreML and INT8 on ONNX Runtime are not interchangeable, and a model evaluated on one can drift on the other. When that runtime-specificity becomes a project-wide cost, the decision shifts to a different compression choice entirely — covered in distillation vs quantisation for multi-platform edge inference. That’s not a recipe. It’s the minimum structure required to avoid making precision decisions based on vibes — in either direction. Neither “FP8 is always fine” nor “FP32 is always required” survives contact with the actual task-specific reality. The only position that holds up is “evaluate, then decide.” FAQ When should I choose distillation over quantisation for edge inference? When you have more than one target runtime and consistent quality across them matters more than peak per-target throughput. Distillation reduces architecture complexity and the resulting model is portable; quantisation tunes precision to a specific runtime and requires separate validation per target. Why does INT8 quantisation behave differently on CoreML, ONNX Runtime, and WebGL — and what does that mean for QA? Each runtime implements quantised operators with different calibration ranges, rounding rules, and kernel fusions. The same INT8 model can produce measurably different outputs across them, so QA must validate per runtime — the work scales linearly with the number of targets. How many edge platforms before distillation’s portability advantage outweighs quantisation’s compute savings? There’s no fixed threshold, but the inflection point is typically when per-target validation cost exceeds the compute savings from quantisation. For most teams that happens at two or three platforms. What quality variation should I expect across CoreML, ONNX Runtime, and TensorRT for the same quantised model? The variation is task-dependent and runtime-dependent — that’s the point of this article. Headline metrics may match closely while tail behaviour, calibration, or per-class accuracy diverge. Measure on the runtime you ship on, with metrics that capture the failure modes you care about. How do I evaluate model-compression options against my deployment matrix without re-validating per platform? You can’t fully avoid per-platform validation when quantising, because the numerical behaviour is genuinely runtime-specific. What you can do is pick a compression strategy — distillation — whose output is portable, so the validation work is shared across targets rather than multiplied by them. Where do ONNX models fit in a multi-platform pipeline — and what are the real performance-vs-portability tradeoffs? ONNX is a portable graph format, but portable graphs don’t guarantee portable numerics under quantisation. An ONNX model in FP32 or BF16 behaves consistently across runtimes; the same model quantised to INT8 does not. Treat ONNX as a deployment lane, not as a guarantee of equivalence. LynxBench AI encodes this discipline — reporting performance at each precision format separately, so task-specific precision sensitivity can inform hardware selection rather than being abstracted away.
