What does deploying a TTS model to three platforms actually require? In a text-to-speech inference optimisation project on edge we ran, separate INT8 quantisation of the same TTS model for CoreML and ONNX Runtime produced audible quality differences at specific phoneme transitions — clean on iOS, a noticeable artefact on Android. The bug was not in the model. It was in the assumption that “INT8” meant the same numerical operation on both runtimes (operational measurement from that deployment; the divergence appeared after the first cross-platform user-acceptance round, not in unit testing). This is the failure mode every multi-runtime TTS deployment hits if the compression decision is made per platform after training rather than as an architectural decision before it. A text-to-speech model that runs satisfactorily in a research environment needs to produce consistent, real-time output on iOS (CoreML), Android (ONNX Runtime), and browser (WebGL/ONNX.js) targets. This is not a deployment engineering problem in the conventional sense — it is an architecture problem that must be solved before model training is complete. The three runtimes process floating-point operations differently, apply different quantisation schemes if model compression is used, and have different memory and throughput constraints per device tier. A model that is optimised for one runtime and then ported to the others will behave consistently on the primary target and diverge on the secondary ones. For a TTS model, divergence manifests as quality differences at the quality-compression boundary: a phoneme transition that the model handles correctly on CoreML may produce an audible artefact on the ONNX Runtime version because the two quantisation implementations apply different precision reduction rules to that specific operation. Why distillation outperforms per-platform quantisation for multi-runtime TTS The distillation vs quantisation decision is particularly clear for TTS models deployed across three or more runtimes: Quantisation per platform requires: (1) a CoreML quantisation pass with INT8 or float16 precision; (2) an ONNX Runtime quantisation pass with its own INT8 scheme; (3) a WebGL-compatible quantisation pass that avoids operations the runtime emulates slowly; and (4) validation of audio quality on each platform separately. For TTS, audio quality validation is domain-specific — perceptual quality metrics like MOS (Mean Opinion Score) and PESQ (Perceptual Evaluation of Speech Quality) must be measured on each platform’s output. Three platforms means three independent quality validation cycles, and quality may diverge between them at the compression boundary. Distillation to a portable model requires: (1) training a smaller student model using knowledge distillation from the larger teacher; (2) exporting the student model once to ONNX; (3) importing the ONNX model into CoreML via CoreML Tools’ ONNX import path; and (4) validating quality once against a reference. The same model artefact runs on all three platforms. Quality at the compression boundary is determined by the distillation training, not by the runtime’s quantisation scheme, and it is consistent across platforms. The TTS distillation architecture In the text-to-speech inference optimisation work on edge we carried out for a mobile application requiring real-time speech synthesis, the production requirement was consistent speech quality across iOS, Android, and desktop browser targets within a latency budget that excluded heavy streaming synthesis approaches. The starting point was a full-size TTS model with high output quality but inference latency significantly above the real-time threshold on mid-range mobile devices. Three compression strategies were evaluated: Approach iOS latency Android latency Browser latency Quality impact INT8 quantisation per platform Within budget Within budget Borderline Audible quality divergence between iOS and Android at some phoneme transitions float16 quantisation (ONNX) Within budget Within budget Exceeds budget on low-end devices Consistent quality, but browser performance insufficient Knowledge distillation (smaller architecture) Within budget Within budget Within budget on all tested devices Consistent quality across all platforms; marginal quality reduction vs full-size model The distilled model used a reduced number of encoder and decoder layers with wider hidden dimensions relative to the full-size model — a dimension configuration that maintained prosody and naturalness at the cost of some fine-grained phoneme distinction, which was acceptable for the application. The teacher-student training used a combination of output distribution matching (KL divergence loss on the output spectrogram distribution) and intermediate layer alignment, which preserved the phoneme boundary behaviour that had been problematic in the quantised versions. ONNX as cross-platform deployment architecture The deployment pipeline for the distilled model treated ONNX as the canonical deployment format rather than as a one-time conversion target. This meant: Version-controlled ONNX export. Each training checkpoint was exported to ONNX as part of the model validation pipeline. The ONNX model version was tracked alongside the training checkpoint version, and the ONNX export was validated against the training environment output before the model advanced to the device testing phase. CoreML import via ONNX. The iOS deployment used CoreML Tools’ onnx.convert() path to import the ONNX model into CoreML format. This maintained a single model artefact (the ONNX file) as the source of truth and eliminated the need for a separate CoreML training or export step. Runtime compatibility testing. The ONNX model was tested against multiple ONNX Runtime versions to validate that the exported operations were supported consistently. Operations that required ONNX opset versions above the deployment target’s support level were identified and replaced with equivalent operations in the lower opset. The result was a deployment pipeline where a single distilled ONNX model produced consistent TTS output on all three target platforms, validated against a single quality baseline rather than three separate ones. The latency on the lowest-specification test devices in the deployment cohort was within the real-time synthesis budget. The runtime portability implication The ONNX Runtime and CoreML are not equivalent execution environments. Specific neural network operations (custom attention mechanisms, non-standard activation functions, complex control flow) may be supported on one and not the other, or supported with different numerical precision. This means model architecture decisions made early in training — before the cross-platform deployment requirement has been specified precisely — can create porting problems later. The equivalent of the device capability baseline for multi-runtime TTS deployment is a runtime operation compatibility audit: before finalising the model architecture, validate that all operations used by the model are supported natively (not emulated) by all target runtimes at their target precision level. Operations that are emulated add disproportionate latency on low-capability devices and are a common source of budget overruns in cross-platform inference projects. The operation compatibility audit checklist The audit is the lowest-cost step in the entire cross-platform deployment workflow and the one most often skipped. The checklist below names the operations and runtime properties that should be validated before architecture is locked in. Each item is a yes/no check against the documented opset and feature support of each target runtime version. # What to check Where it bites 1 ONNX opset version supported by the lowest target runtime. Identify the minimum opset version supported by the oldest ONNX Runtime version in the device cohort, the WebGL/WebGPU ONNX.js version, and the CoreML Tools ONNX import path. The model must export to that opset or below. A model exported at opset 17 will fail to load on a runtime that supports up to opset 14. Discovered late, this forces either an export downgrade (potentially regressing quality) or dropping the affected device tier. 2 Custom attention mechanisms. Validate that the attention pattern (standard scaled dot-product, multi-query, grouped-query, sliding-window, sparse) maps to a supported operation on each runtime. CoreML and ONNX Runtime have diverged on which attention variants they implement natively. Custom attention falls back to a per-element implementation and inference latency multiplies by 5–20× on the affected runtime. 3 Activation functions outside the standard set. ReLU, sigmoid, tanh, GELU (approximate and exact variants), SiLU/Swish are widely supported. Custom activations (Mish, Snake, learned activations) are not. Custom activation falls back to a graph of primitive operations, with both latency and numerical precision implications. 4 Normalisation operations. LayerNorm and BatchNorm are universally supported. RMSNorm, GroupNorm, and conditional normalisation variants are not uniformly supported across CoreML versions. Unsupported normalisation breaks model export entirely or produces a fallback that violates the latency budget. 5 Dynamic shape support. TTS models often have variable-length input (text) and variable-length output (audio frames). Validate that each runtime supports dynamic axes for the relevant tensor dimensions. Static-shape-only runtime requires either bucketed input padding (latency overhead, quality implications at boundary) or per-shape model export (deployment artefact explosion). 6 Control flow operations. If, Loop, and Scan operations are supported variably across runtimes. Autoregressive decoding loops that use Loop for variable-length output may not export cleanly. Loop fallback runs the body on host code rather than the runtime, producing per-step host-runtime transition overhead. 7 Numerical precision per operation. Validate that float16 (or the chosen reduced precision) is supported for every operation in the model on every target runtime. Some runtimes promote to float32 for specific operations, with bandwidth and latency implications. A model designed around float16 throughput finds half its operations promoted to float32 silently, with the latency budget exceeded as a consequence. 8 Tensor layout assumptions. NCHW vs NHWC layout, and per-runtime conversion costs for layout transitions. CoreML prefers a specific layout that may not match the ONNX export default. Hidden layout transposes inside the inference graph add latency that does not appear in profiling against the development environment. 9 Batched vs streaming inference paths. TTS in production may run as either batched synthesis (full sentence at once) or streaming synthesis (chunk by chunk). Validate that the chosen path is performant on each runtime; streaming synthesis exposes per-chunk overhead that batched synthesis amortises. A model architected for batched synthesis on the development environment may be unworkable as streaming synthesis on the deployment runtime, forcing a late architecture change. 10 CoreML version coverage of the iOS device cohort. Model export to a CoreML version requires that minimum iOS version on the deployed devices; older iOS versions cannot load newer CoreML format versions. A model targeted at the latest CoreML format excludes a meaningful fraction of the active device base. A model architecture that passes this audit cleanly can be developed against any one runtime and ported to the others with minimal surprise. A model that fails one or more items at the audit stage should be modified at the architecture stage rather than at the deployment stage — the modification is cheaper before training is complete than after. What remained imperfect The distilled cross-platform TTS deployment met its quality and latency targets, but two limitations remain worth naming: First, the distilled model is measurably lower quality than the full-size teacher on certain speech features — specifically, fine-grained prosody control on long sentences and the realism of expressive speech (emphatic delivery, emotional inflection). The application accepted this trade-off because the use case prioritised latency and platform consistency over peak quality, but a deployment that cannot accept any quality reduction would need a different approach: a server-side fallback for the highest-quality requests, a per-platform optimised pipeline at the cost of consistency, or a larger student model at the cost of latency on lower-tier devices. Second, even with ONNX as the canonical deployment format, the runtime versioning matrix continued to require maintenance over time. New ONNX Runtime versions, new CoreML versions, and new browser engine versions periodically introduced changes in operation support, numerical behaviour, or default precision. Each required a regression test pass against the deployment cohort. The single-artefact deployment reduced the per-release cost compared to a per-platform-quantised approach, but it did not eliminate the cross-platform validation cost — it shifted that cost from the deployment phase to the maintenance phase. For teams approaching this architecture for the first time, a GPU and Inference Optimisation Assessment evaluates the model architecture and compression strategy against the runtime compatibility requirements of the target deployment environment.
