Ask most teams for their model metrics and you get one number: validation accuracy. That works when the model ships to one runtime. The moment the same compressed model deploys to CoreML on iOS, ONNX Runtime on an edge box, and WebGL in a browser, that single number stops describing what users actually experience.

The naive framing treats “model metrics” as something you measure once, on a held-out validation set, before deployment. The expert framing treats metrics as a matrix — quality, latency, and size measured per target — and treats the variance across those targets as its own first-class metric. That variance is where multi-platform deployments quietly fail, and it is invisible to any single accuracy number.

Which metrics actually matter across CoreML, ONNX Runtime, and WebGL?

A model that hits 94% accuracy in your PyTorch training loop does not necessarily hit 94% once it has been converted to CoreML, exported to ONNX and run under ONNX Runtime, or compiled down for WebGL execution in a browser. Each conversion path applies its own operator fusions, its own numeric handling, and — critically — its own quantisation behaviour. The number you measured once was measured on exactly none of the runtimes your users will hit.

For multi-platform edge, three metric families matter, and each has to be captured per target:

• Quality — task accuracy on the converted, on-device model, not the reference checkpoint. For a detector this is mAP; for a classifier, top-1/top-5; for text-to-speech, an intelligibility or MOS-proxy score.
• Latency — end-to-end inference time on the target’s actual runtime and hardware, including the cost of the first inference (warm-up and kernel compilation) versus steady-state.
• Size — the deployed artifact size after conversion, which differs across formats because each runtime packs weights and metadata differently.

The single most under-measured number is the cross-target quality spread: the gap between the best- and worst-performing runtime for the same source model. Teams that only track a mean accuracy never see it. It is the metric that decides whether a multi-platform deployment is safe to ship.

This is a different question from the one we cover in ML model metrics that explain GPU bottlenecks versus metrics that mislead — that article is about reading a single deployment’s compute behaviour. Here the subject is the divergence between deployments of the same model.

Why one validation-set accuracy misleads for a compressed model

Compression is the divergence point. When you distil a model — training a smaller student to mimic a larger teacher — the student’s weights are learned end-to-end, and the resulting model tends to hold a single quality number across runtimes, because the conversion step is mostly a faithful re-expression of the same float weights. The runtime differences are real but small.

Per-platform quantisation behaves differently. Quantising a model to INT8 (or lower) is not one operation applied identically everywhere. CoreML’s quantisation, ONNX Runtime’s quantisation, and a browser-side WebGL path each make their own calibration and rounding choices. The same source model, quantised down three different pipelines, produces three different accuracy figures — and the spread between them is not something you can predict from the reference checkpoint. In our experience porting inference paths across runtimes, this spread is the number that surprises teams in production QA, because nothing in a single-runtime workflow forces you to look at it.

That is the trap in “we validated the model.” You validated a model — the float reference — not the three quantised artifacts your users will run. If you only measure accuracy once, the cross-target variation stays hidden until it shows up as a quality regression on one platform and a support ticket queue on another.

The correction is mechanical: measure quality on each converted artifact, on each target runtime, and record the spread. The measurement is task-agnostic. The same quality/latency/size approach applies whether the model is a TTS voice, an object detector, or a classifier — which is why a metric matrix built for one workload transfers cleanly to the next. Getting the conversion itself right often comes down to the build path; the compilation flags for multi-platform edge inference determine a good deal of what the numbers turn out to be, and dividing work across CPU, GPU, and WASM targets changes which target owns which part of the latency budget.

A per-target metric table you can extract

The structured artifact for a multi-platform deployment is a per-target metric table. It converts a vague “is quality acceptable?” into a bounded decision. Here is an illustrative shape — the numbers are examples to show the structure, not a benchmark:

Target	Quality (top-1)	p50 latency	p95 latency	Artifact size	Quality delta vs reference
Reference (FP32)	94.1%	—	—	48 MB	0.0 pp
CoreML (INT8)	93.4%	11 ms	18 ms	13 MB	−0.7 pp
ONNX Runtime (INT8)	92.1%	9 ms	15 ms	12 MB	−2.0 pp
WebGL (FP16)	91.6%	34 ms	61 ms	24 MB	−2.5 pp
Cross-target spread	1.8 pp	—	—	—	—

Read the last row first. The cross-target quality spread — the gap between the best and worst deployed runtime, here 1.8 percentage points — is the metric that gates the decision. If your product tolerance says a deployment is acceptable when cross-target accuracy spread stays under, say, 2 pp, this table passes; if the threshold is 1 pp, it fails and one target needs work before ship.

The threshold is a product decision, not a technical one, but the table makes it decidable. Set the acceptable spread up front. Then the per-target measurement tells you, before production, whether you are inside it. That is the difference between quantifying the hidden validation cost and discovering it through QA cycles.

How the compression choice reads off the table

The distillation-versus-quantisation decision is where this table earns its place. The two strategies produce structurally different tables:

• Distillation tends to produce a tight spread. The student model carries roughly one quality number that survives conversion, because the compression happened in training rather than in each runtime’s post-hoc quantiser. You pay for it up front in training compute and effort.
• Per-platform quantisation produces a wider spread. It is cheap to apply and gives you strong per-target compute and size savings — but the savings can be offset by the cost of validating each target separately when the accuracy diverges.

The trade-off is concrete. For a three-platform deployment, per-platform quantisation’s compute and size wins have to be weighed against the possibility of running roughly three separate QA cycles to certify each target’s quality independently — the hidden validation cost that a single accuracy number never exposes. The metric table is what lets you compare the two strategies on the same axis: does the quantised spread stay under your threshold, or does distillation’s tighter spread justify its higher training cost? This is an observed pattern from porting work, not a fixed rule — the answer depends on your tolerance and the model, which is exactly why you measure rather than assume.

For teams instrumenting this in a live telecom or media edge pipeline, the metric matrix is the same instrument that our GPU and inference optimization work uses to check whether a chosen compression strategy is justified by its measured spread, and it feeds naturally into the broader question of what to track once models are running — covered in what to monitor for GPU inference workloads. In edge deployments across mobile networks and set-top boxes, the same per-target discipline applies to the telecom side of the pipeline, where the runtime mix is rarely under one team’s control.

FAQ

What matters most about machine learning model metrics in practice?

In practice, a model metric is a measurement of a specific model property — quality, latency, or size — on a specific runtime and input distribution. For a single-runtime deployment you can get away with one accuracy number, but for a multi-platform edge deployment each metric must be measured per target, because conversion and quantisation change the numbers on every runtime. The useful unit is a metric matrix, not a scalar.

Which metrics actually matter when the same model is deployed to multiple edge targets like CoreML, ONNX Runtime, and WebGL?

Quality (task accuracy on the converted on-device model), latency (end-to-end inference time on the target’s real runtime, including warm-up), and artifact size — each captured per target. On top of those, the cross-target quality spread is the metric that most often decides whether the deployment is safe, and it is the one teams most often fail to measure.

Why can accuracy measured once on a validation set be misleading for a compressed multi-platform model?

Because you validated the float reference, not the three quantised artifacts your users will actually run. Each runtime applies its own quantisation, fusion, and numeric handling, so the same source model produces different accuracy figures per target. A single validation number hides that spread until it appears as a per-platform quality regression in production QA.

How do I measure cross-target quality variance, and what threshold makes it acceptable?

Run the same evaluation set through each converted artifact on its target runtime, record per-target quality, and take the gap between the best and worst as the spread. The acceptable threshold is a product decision — for example, ship only if the spread stays under a stated bound like 1–2 percentage points — set before deployment so the measurement gates the release rather than describing a failure after it.

How do latency, model size, and quality metrics trade off differently under distillation versus per-platform quantisation?

Distillation tends to hold one quality number across runtimes (tight spread) at the cost of higher training effort. Per-platform quantisation is cheap and gives strong per-target compute and size savings but produces a wider quality spread, whose validation cost can approach one QA cycle per target. The metric table lets you weigh the quantised spread against distillation’s tighter but more expensive result on the same axis.

How do I build a per-target metric table that feeds the distillation-vs-quantisation decision?

List every deployment target as a row, add columns for quality, p50/p95 latency, and artifact size measured on each target’s real runtime, and compute a cross-target spread row. Set your acceptable spread threshold first, then read the table: if quantisation’s spread stays under the threshold you take the compute savings; if it exceeds it, distillation’s tighter spread may justify its higher training cost.

The open question is rarely “which compression method is best” in the abstract — it is whether your measured cross-target spread stays under the threshold your product can tolerate, and that is only answerable once the per-target table exists. If your deployment plan still rests on one accuracy number, the first thing to build is the matrix that shows you what that number is hiding.