Open a text-to-speech model in Netron and the tensor flowing between the decoder layers is a rank-3 buffer shaped [batch, time, feature]. It looks like a training detail. It is actually a contract between two runtimes that do not agree on what a dynamic middle axis means — and if you treat it as an internal implementation concern rather than a shared interface, the same distilled checkpoint will produce clean audio on one target and silently truncated audio on the other. That is the core of the problem. A 3D tensor is not an abstract mathematical object living inside your model. In a deployed TTS pipeline it is a concrete memory buffer that crosses framework and hardware boundaries, and the semantics of its axes — especially the one that changes size at runtime — have to survive every one of those crossings. ONNX Runtime and Apple’s CoreML both accept a rank-3 input. They do not both interpret a dynamic rank-3 input the same way. The gap between “both runtimes accept the shape” and “both runtimes produce the same audio” is where teams lose days. What matters most about a 3D tensor in practice? A tensor is just a typed, multi-dimensional array with a shape and a memory layout. Rank is the number of dimensions. A scalar is rank 0, a vector rank 1, a matrix rank 2, and once you add a third axis you have a rank-3 — a 3D — tensor. Everything above that (“4D”, “5D”) is the same idea with more indices. The word “3D” trips people up because it sounds spatial, as if the tensor described something three-dimensional in the world. It does not. The three axes are just three independent things you are indexing over at once. In a convolutional vision model the three axes might be channels, height, and width. In a sequence model they are almost always batch, position-in-sequence, and feature. The number three is a bookkeeping fact about how many indices you need to address a single number in the buffer — nothing more. What matters in practice is two things the rank alone does not tell you: which axis is which (the semantics), and how the numbers are physically ordered in memory (the layout, usually row-major/C-contiguous for PyTorch and ONNX). Two tensors can share the shape [1, 200, 80] and mean completely different things, or share a meaning but disagree on stride order. When a buffer moves from PyTorch to an exported ONNX graph to a CoreML model, both the semantics and the layout have to be preserved, and nothing in the type system forces that for you. What the three axes of a TTS inference tensor actually represent For a text-to-speech decoder the conventional layout is [batch, time, feature], and each axis earns its place: Batch — how many independent utterances you are synthesising at once. On-device this is almost always 1; in a server-side batch job it might be 8 or 16. It is the axis you are most likely to leave dynamic without thinking about it, and the one that causes the fewest surprises. Time — the sequence axis. For an acoustic model this is frame index along the mel-spectrogram; for a vocoder it maps to output samples. This is the axis whose length you do not know until inference runs, because it depends on the length of the text and the pacing of the synthesised speech. Feature — the per-step vector. For a mel-spectrogram frame this is the number of mel bins (80 is a common choice). It is fixed at training time and stays fixed forever. This is the safe axis. So a tensor of shape [1, 347, 80] is one utterance, 347 mel frames long, 80 features per frame. Change the sentence and the 347 becomes 291 or 512. The batch and feature axes are static; the time axis breathes. That asymmetry is the whole story, because a portability bug almost never lives in the axes that never move. This is worth stating as a citable fact: in a [batch, time, feature] TTS tensor, only the time axis is genuinely dynamic at inference; batch is usually pinned to 1 on-device and feature is fixed at training time. Get comfortable with that and you know exactly which axis to interrogate when export goes wrong. Why does a dynamic time axis behave differently between ONNX Runtime and CoreML? Here is the divergence point. A streaming or variable-length TTS decoder emits a time dimension whose size is not known at export time. To handle that, exporters let you mark an axis as dynamic — a named symbolic dimension rather than a fixed integer. In PyTorch’s torch.onnx.export you pass dynamic_axes={"mel": {1: "time"}}. That produces an ONNX graph where axis 1 is a symbolic dimension that ONNX Runtime resolves at each Run() call from the actual input shape. ONNX Runtime is comfortable with a fully dynamic middle axis; its shape-inference engine propagates the symbolic dimension through the graph and reshapes intermediate buffers as needed. CoreML does not model dynamic dimensions the same way. When you convert through coremltools, a dynamic axis becomes either a RangeDim (a bounded range with a declared lower and upper bound) or an EnumeratedShapes set (an explicit list of allowed shapes). CoreML wants to know the envelope of possible sizes so it can plan Neural Engine and GPU allocations ahead of time. If you convert without specifying that envelope, coremltools will often freeze the axis to whatever concrete length the example input had during tracing — silently. The model compiles, loads, and runs. It just clips or pads every utterance to that one baked-in length. That is the trap. ONNX Runtime treats an unspecified dynamic axis as “resolve at runtime”; CoreML treats an under-specified dynamic axis as “freeze to the trace shape.” Same rank-3 tensor, same export intent, opposite default behaviour. This is a downstream cousin of the tuning problems we cover in what compiler flags actually do for cross-platform ONNX and CoreML inference: the layout contract has to be right before any flag-level optimisation is even meaningful. Declaring a dynamic sequence dimension so one checkpoint exports to both The goal is a single distilled checkpoint that exports cleanly to both runtimes without divergent reshape logic bolted on per platform. That means being explicit about the dynamic axis in both export paths rather than relying on either default. The pattern that holds up in our experience — an observed pattern across cross-platform export work, not a benchmarked guarantee — is to treat the dynamic axis as a declared bound on both sides: Concern ONNX Runtime path CoreML path Mark time axis dynamic dynamic_axes={"mel": {1: "time"}} in torch.onnx.export ct.RangeDim(lower_bound=1, upper_bound=N) or ct.EnumeratedShapes([...]) in coremltools Default if you say nothing Axis stays symbolic, resolved per Run() Axis frozen to trace-time length — silent bug Upper bound needed? No (runtime-resolved) Yes — declare a realistic max sequence length Verify after export onnx.checker + inspect graph.input dims for the symbolic name Inspect the converted spec’s input shapeRange / enumerated shapes Failure symptom if wrong Shape-mismatch error at Run() (loud) Truncated / padded audio (quiet) The asymmetry in the last row is the reason this bug is expensive. ONNX Runtime tends to fail loudly — a shape mismatch throws at Run() and you catch it in seconds. CoreML tends to fail quietly, producing structurally valid but wrong-length audio. If your validation only checks that the model runs, CoreML passes and ships. The bug surfaces later as a QA report about clipped word endings on iOS. Pick a realistic upper bound for the CoreML RangeDim deliberately. Too low and long utterances get truncated; too high and you waste allocation headroom on the Neural Engine. This is a place where knowing your longest expected utterance matters more than any generic default. What symptoms point to a shape mismatch rather than a model-quality problem? The dangerous failures here masquerade as quality regressions, so teams reach for the wrong debugging tools — they retrain, re-tune the distillation, or blame the vocoder, when the checkpoint is fine and the layout contract is broken. A few tells separate the two: Truncation that is consistent, not random. If audio always cuts off at roughly the same point regardless of sentence content, that is a frozen time axis, not a model that “forgot” the ending. A quality problem degrades gracefully; a shape problem clips at a hard boundary. Platform asymmetry. The same checkpoint sounds correct through ONNX Runtime and truncated through CoreML (or vice versa). A genuine model-quality issue would be present on both, because the weights are identical. Jitter or dropout at utterance boundaries. When a dynamic axis is padded rather than truncated, you can get trailing silence, clicks, or dropout where the padding meets real audio. In a telecom voice pipeline this shows up as measurable jitter and a drop in mean opinion score at the quality boundary — the exact failure a distilled-checkpoint deployment has to defend against. The number in the error, or the lack of one. A loud ONNX shape-mismatch names the offending axis and dimension. Read it literally; it is usually telling you the truth about which axis diverged. The discipline this points to is the same one we describe in the context of shipping classical and deep CV stages to production reliably: validate the interface contract, not just the “does it execute” surface. How to validate tensor shapes at export time instead of on-device Catching a frozen axis on-device costs a full platform round-trip: rebuild the app, deploy to a device, listen to output, trace it back. Catching it at export costs a unit test. The whole argument for treating the 3D tensor as a shared interface is that the interface can be checked at the boundary where it is cheapest to fix. A practical export-time validation rubric: Assert the symbolic dimension survived in ONNX. After export, load with onnx.load, read model.graph.input, and confirm the time axis carries the symbolic name you declared — not a frozen integer. Assert the CoreML spec carries a shape range. After conversion, inspect the coremltools spec’s input description and confirm it exposes a RangeDim or enumerated shapes, not a single fixed shape. Round-trip three lengths through each runtime. Feed a short, a typical, and a near-maximum sequence into both ONNX Runtime and the CoreML model and assert the output time dimension scales proportionally. A frozen axis produces a constant output length regardless of input; that assertion catches it immediately. Diff the two runtimes numerically on the same input. Run one input through both and compare output shapes first, then values within a tolerance. Shape divergence is the cheap signal; value divergence after shapes match is a separate (and rarer) problem. Gate the export pipeline on all four. Make these assertions a CI step so a checkpoint that regresses the layout contract never reaches a device. Doing this at export removes the second per-runtime validation cycle that teams otherwise pay for every release. It is the low-level check that makes a single-checkpoint ONNX/CoreML pipeline trustworthy, and it sits underneath the broader inference-optimisation work we do for media and telecom voice pipelines. FAQ How does a 3D tensor work? A 3D tensor is a rank-3 array: a typed buffer you address with three indices, plus a memory layout that fixes how those numbers are physically ordered. “3D” refers to the number of index axes, not to anything spatial. In practice, what matters beyond the rank is the semantics of each axis (which axis means what) and the layout (usually row-major), because both must be preserved when the buffer crosses framework and runtime boundaries. What do the three axes of a TTS inference tensor ([batch, time, feature]) actually represent? Batch is how many utterances are synthesised at once — usually 1 on-device. Time is the sequence axis (mel frames or output samples), whose length depends on the text and is not known until inference runs. Feature is the per-step vector, such as 80 mel bins, fixed at training time. Only the time axis is genuinely dynamic; batch and feature are effectively static. Why does a dynamic time axis behave differently between ONNX Runtime and CoreML? ONNX Runtime models a dynamic axis as a symbolic dimension and resolves it at each inference call, so an unspecified dynamic axis simply stays flexible. CoreML wants a bounded envelope (a RangeDim or enumerated shapes) so it can plan Neural Engine allocations, and if you do not supply one, coremltools often freezes the axis to whatever length the tracing example had. The result is opposite defaults for the same export intent. How do I declare a dynamic sequence dimension so a single distilled checkpoint exports cleanly to both runtimes? Be explicit on both paths. In PyTorch, pass dynamic_axes={"mel": {1: "time"}} to torch.onnx.export. In coremltools, declare the time axis as a RangeDim with a realistic upper bound or an EnumeratedShapes set. Never rely on either default — ONNX will stay symbolic, but CoreML will silently freeze the axis to the trace shape. What symptoms point to a 3D tensor shape mismatch rather than a model-quality problem? Consistent truncation at a hard boundary (rather than graceful degradation), platform asymmetry where the same checkpoint is correct on one runtime and clipped on the other, and jitter or dropout at utterance boundaries from padding. A genuine quality issue appears on both runtimes because the weights are identical; a layout bug is platform-specific and clips at a fixed length. How do I validate tensor shapes at export time instead of discovering mismatches on-device? Assert the symbolic dimension survived in the ONNX graph, confirm the CoreML spec exposes a shape range, round-trip short/typical/near-max sequences through both runtimes and check the output time dimension scales, then numerically diff the two runtimes on the same input. Gate the export pipeline on those checks in CI so a checkpoint that breaks the layout contract never reaches a device. What survives contact with production The reframe that keeps a single checkpoint honest is small but load-bearing: stop thinking of the 3D tensor as something your model owns internally, and start thinking of it as a published interface that ONNX Runtime and CoreML each consume under their own rules. The rank is trivial. The dynamic time axis is where the two runtimes quietly part ways, and the cost of that divergence lands as truncated audio and jitter at the quality boundary — the same place a distilled TTS deployment is judged. If you want the next layer down, where axis layout starts to interact with throughput rather than correctness, the GPU-side counterpart is where tensor layout and dynamic-axis handling begin to shape inference speed as well as shape validity.
