Dropping a computer vision model to 4-bit floating point looks like a clean win: half the memory of FP8, more throughput, a smaller edge footprint. The trap is treating the conversion as a format change rather than what it actually is — a data-quality event. FP4’s tiny mantissa and narrow dynamic range don’t compress every part of the model equally. They compress the activation distributions that encode rare features hardest, and the classes your model already saw least often — rare defects, unusual lighting, edge conditions — are exactly the ones that collapse first. That’s the core of it. A model can score well on FP4 during validation and still degrade in the field, because the two things that decide FP4’s fate — calibration and drift — are invisible unless you measure them on purpose. This article is about why that happens and what to measure before you commit. What’s worth understanding about 4-bit floating point first? FP4 packs a floating-point value into four bits: one sign bit, a small exponent field, and a mantissa of one or two bits depending on the variant (E2M1 is the common shape for CV weights and activations). Compared to FP8, you keep the idea of floating point — an exponent gives you dynamic range across several orders of magnitude — but the mantissa is so short that the representable values within any exponent band are extremely coarse. Between two adjacent FP4 values sits a wide gap that every real number has to round into. In practice that means FP4 is not a uniform “everything gets a bit fuzzier” operation. The rounding error is proportional to the magnitude of the value, and the sparse mantissa means whole regions of an activation histogram get mapped onto the same handful of representable points. Values that cluster tightly around common patterns survive because they land near a representable point. Values that sit in the long tail — the activations that fire for the rare cases — get quantized into oblivion or merged with their neighbours. The mechanics of how FP4 encodes a number and where it fits for edge inference are worth reading alongside this, because the encoding is what drives the precision behaviour we’re about to describe. The practical takeaway: FP4 is a lossy transform whose loss is not uniformly distributed across your data. That asymmetry is the whole story. How does FP4 differ from INT4 and FP8, and when should each be used? The three formats occupy different points on the range-versus-resolution trade-off, and the right choice depends on what your activation distributions look like — not on which number is smallest. Format Bits What it buys Where it fits Where it hurts FP8 (E4M3 / E5M2) 8 Wide range and usable resolution; forgiving of miscalibration Default first stop for CV quantization; models with heavy-tailed activations Twice the memory of FP4; may leave throughput on the table on hardware with native FP4 paths FP4 (E2M1) 4 Half the memory of FP8; high throughput on FP4-capable accelerators Latency- or memory-bound edge deployment where you’ve proven per-class accuracy holds Coarse resolution collapses long-tail activations; unforgiving of calibration error INT4 4 Uniform quantization steps; simple, well-tooled Weights with roughly symmetric, bounded distributions Fixed step size wastes precision on peaky distributions; no dynamic range across scales The decision rule we apply in practice: FP8 is the safe default. Move to FP4 only when a memory or latency budget forces it and you have measured that the accuracy loss lands where you can afford it. INT4 versus FP4 is a distribution question — INT4’s uniform steps suit weights that fill a bounded range evenly, while FP4’s exponent handles activations that span several orders of magnitude but pays for it in mantissa resolution. Neither 4-bit format is a free upgrade from FP8; both are trades you should be able to justify with a per-class number. This is a calibration and validation concern more than a hardware concern. If you’re choosing quantization primarily to hit a hardware target, the runtime and compiler side of the decision — how ML compilers and inference runtimes optimize a quantized CV model — matters, but it won’t tell you whether your rare classes survived the conversion. Why does 4-bit quantization degrade rare classes more than common ones? Quantization fits a fixed set of representable values to the distribution of numbers flowing through your network. Calibration decides where those values sit by observing a sample of activations and choosing a range to cover them. The optimizer that picks the range is trying to minimise total error across the calibration set — and total error is dominated by common cases, because common cases contribute most of the samples. The consequence is structural, not incidental. The quantization range gets fit to the bulk of the distribution. The long tail — the activations that only fire for rare defects or edge conditions — either falls outside the covered range and gets clipped, or falls into a region so coarsely quantized that distinct rare-feature signals become indistinguishable. The features your model relies on to separate a rare class from the background are precisely the low-frequency signals that FP4’s short mantissa cannot resolve. This is why a global accuracy number lies to you. A model that loses a handful of accuracy points overall may have lost almost nothing on its common classes and a large fraction of its recall on the rare ones. In an inspection setting, that shows up as a defect detector that still catches the common scratch but stops catching the rare crack — the failure that actually matters. The same asymmetry is why per-class confidence scores behave differently after quantization: the scores on rare classes get noisier and less separable, even when the aggregate metric looks fine. There’s an uncertainty framing here worth naming. Quantization error is a form of injected noise, and it interacts differently with the two kinds of uncertainty in a model — the point is developed in our discussion of aleatoric versus epistemic uncertainty in production ML. FP4 error behaves like added aleatoric noise concentrated exactly where the model was already least confident. How does calibration data quality determine whether an FP4 model holds accuracy? Calibration is the single decision that separates an FP4 conversion that holds from one that quietly fails. The quantization range is fit to whatever activations the calibration set produces. If that set is a representative sample of your production distribution — including the rare classes and edge conditions at something close to their real frequency — the fitted range covers the signals that matter. If it’s a convenient training sample that under-represents the tail, the range gets fit to the easy part of the distribution and the tail gets clipped. The failure mode is that the two look identical at validation time. A calibration set drawn from the same convenient sample as your validation set will produce a model that validates beautifully and degrades in deployment, because both the calibration and the validation shared the same blind spot. This is a data-distribution problem wearing a numerical-format costume, which is why we treat low-precision quantization behaviour as part of a data quality audit rather than a compiler setting. Concretely, calibration data quality means three things: coverage (does the set contain the rare classes and edge conditions at all), representativeness (are they present at a frequency that lets the range-fitting optimizer weight them), and freshness (does the set reflect the current production distribution, not last quarter’s). A calibration set that was representative when you built it can become misrepresentative as the world changes — which is where drift enters. How can low-precision quantization interact with data drift to compound accuracy loss? Here is the compounding effect that makes FP4 riskier in production than in the lab. Quantization freezes a set of representable ranges fit to the distribution you calibrated on. Data drift moves the live distribution away from that. When the two combine, activations that used to land inside the covered range start landing outside it, and the coarse FP4 grid that was already marginal for rare features becomes marginal for a growing slice of ordinary inputs too. Neither effect alone would necessarily trip an alarm. Mild drift on a full-precision model degrades accuracy gradually and detectably. FP4 with good calibration on a stable distribution holds. But drift plus FP4 is nonlinear: the quantization ranges no longer cover the shifted distribution, clipping increases, and the accuracy loss accelerates faster than either factor would predict on its own. And because the loss concentrates on the tail first, it’s the rare-but-important classes that erode while the aggregate metric still looks acceptable. The reason this stays hidden is that it lives in the gap between two monitoring habits. Drift monitoring watches the input distribution. Accuracy monitoring watches the aggregate metric. FP4’s tail collapse falls between them — the input drift is mild enough to look tolerable, the aggregate accuracy holds long enough to look fine, and the per-class recall on rare classes drops without either dashboard flinching. You only see it if you monitor per-class retention against a fixed baseline. This is the same pattern the standard object detection metrics — precision, recall, mAP, IoU — will surface only when you break them out per class rather than reading the headline mAP. How do I measure per-class accuracy retention before committing to FP4? The measurable outcome that justifies FP4 is accuracy retained per bit reduced, tracked against the pre-quantization baseline and broken out by class. Here is the check we run before signing off on a 4-bit conversion. FP4 readiness checklist Establish the FP8 (or FP16) baseline per class. Record precision and recall for every class separately, not just aggregate mAP. This is your reference; without it the FP4 numbers are unanchored. (operational measurement against the deployed baseline model.) Audit the calibration set for tail coverage. Confirm the rare classes and known edge conditions appear at a frequency that lets the range-fitting optimizer weight them. A calibration set that omits the tail cannot produce a range that covers it. Quantize to FP4 and re-measure per class. Compute the per-class delta. Expect the loss to concentrate on the rarest classes — that concentration, not the average, is the number that decides the trade. Set a per-class floor, not a global one. Define the minimum acceptable recall for each class that carries operational cost. In our experience the classes that matter most (the rare defect, the safety-relevant edge case) are the ones that fail the floor while the aggregate passes — an observed pattern across quantization work, not a benchmarked rate. Simulate a drifted distribution. Feed the FP4 model inputs that reflect plausible production shift and re-measure per class. If retention degrades sharply under mild drift, the calibration range is too tight and FP4 is not yet safe to ship. Decide against the budget you started with. FP4 halving memory versus FP8 is only justified when the per-class degradation is measured and acceptable. If the rare-class loss exceeds the operational floor, stay on FP8 — the memory saving does not pay for the missed detections. The trade FP4 offers — roughly half the memory of FP8 in exchange for coarser resolution — is a real and sometimes necessary one on memory- or latency-bound edge hardware. It becomes a good trade only when the per-class cost is on the table before deployment, not discovered after a missed detection in the field. FAQ How does 4-bit floating point work in practice? FP4 stores a value in four bits — one sign, a small exponent, and a one- or two-bit mantissa (E2M1 is the common CV shape). The exponent preserves dynamic range, but the tiny mantissa makes the representable values extremely coarse, so rounding error is large and unevenly distributed. In practice FP4 is a lossy transform whose loss falls hardest on the long tail of an activation distribution rather than uniformly across all inputs. How does FP4 differ from INT4 and FP8, and when should each be used for CV inference? FP8 is the forgiving default — wide range and usable resolution — and should be your first stop. FP4 halves FP8’s memory and runs fast on FP4-capable accelerators, but its coarse resolution collapses long-tail activations, so use it only when a memory or latency budget forces it and you’ve proven per-class accuracy holds. INT4’s uniform steps suit weights with bounded, evenly-filled distributions, while FP4’s exponent suits activations spanning several orders of magnitude at the cost of mantissa resolution. Why does 4-bit quantization degrade rare classes and edge cases more than common ones? Calibration fits the quantization range to minimise total error, which is dominated by common cases because they contribute most samples. The long-tail activations that encode rare defects or edge conditions either fall outside the fitted range and get clipped, or land in coarsely quantized regions where distinct signals merge. The features that separate a rare class from the background are exactly the low-frequency signals FP4’s short mantissa cannot resolve, so rare-class recall drops while aggregate accuracy looks fine. How does calibration data quality determine whether an FP4 model holds accuracy in production? The quantization range is fit to whatever activations the calibration set produces, so a set that under-represents the tail produces a range that clips it. The danger is that a calibration set sharing a blind spot with the validation set produces a model that validates well and degrades in deployment. Good calibration data means coverage of rare classes, representativeness at realistic frequency, and freshness against the current production distribution. How can low-precision quantization interact with data drift to compound accuracy loss after deployment? Quantization freezes representable ranges fit to the calibrated distribution; drift moves the live distribution away from it, so activations increasingly fall outside the covered range and clipping rises. The two combine nonlinearly — accuracy loss accelerates faster than either factor alone — and it concentrates on the tail first. It stays hidden because drift monitoring watches inputs and accuracy monitoring watches aggregates, while the tail collapse falls in the gap between them. How do I measure per-class accuracy retention before committing a CV model to FP4? Establish a per-class precision and recall baseline in FP8 or FP16, audit the calibration set for tail coverage, quantize to FP4 and re-measure the per-class delta. Set a per-class recall floor for every operationally costly class rather than a single global threshold, then simulate a drifted distribution and re-measure. Commit to FP4 only when the rare-class degradation is measured and stays inside the floor — the memory saving does not pay for missed detections. FP4 is not a checkbox on a deployment config; it’s a decision about which of your classes you can afford to see less clearly. Treating low-precision conversion as part of a data quality audit — the same audit that catches an unrepresentative training set — is what keeps the rare-class collapse from surfacing as a missed detection in production rather than a number on a validation report. If you’re weighing the format against a hardware target, start from what your activation distributions actually look like, not from which number is smallest.
