The single precision floating point format, usually written FP32, is the IEEE 754 binary32 encoding. It has 1 sign bit, 8 exponent bits and 23 explicitly stored significand bits. With an implicit leading bit, that gives 24 bits of precision and between 6 and 9 significant decimal digits. The exponent carries a bias of 127, so normal numbers cover exponents from −126 to +127. FP32 is so consistently the default that tutorials and framework configurations treat it as a property of training itself. In fact it is a specific design choice resting on specific assumptions. That reading is the one to correct. FP32 is a well-understood baseline, not a certificate of correctness, and the formats on either side of it trade numerical representation in ways an engineer should choose rather than inherit.
What does the 1+8+23 bit allocation in FP32 actually control?
Each of the three fields controls a different property of the number, and they are worth keeping apart. The value of a normal binary32 number is (−1)^S × 2^(E−bias) × (1 + 2^(1−p) × T). Here S is the sign, E the biased exponent field and T the trailing significand field, so normal numbers always carry an implicit leading 1. That hidden bit is why 23 stored bits deliver a 24-bit significand (p = 24).
The sign bit is the simplest field. IEEE 754 has both +0 and −0, which compare as equal but give different results in some operations. The 8 exponent bits decide scale. The field runs from 0 to 255, but the all-zeros and all-ones patterns are reserved for special numbers. That is why usable normal exponents stop at −126 and +127. The 23 fraction bits decide resolution: how finely the format can tell two neighbouring values apart at a given scale.
A practical way to hold this: exponent bits decide whether a value can be represented at all, and fraction bits decide how close the stored value lands to the one you meant. Every basic operation is computed as if to infinite precision and then rounded to fit the destination format. For binary formats the default rounding rule is round to nearest, ties to even.
Single precision floating-point range, digits and the edges of exactness
FP32’s 24 bits of precision are equivalent to about 7.225 decimal digits for normal values. PyTorch’s documentation describes it as “about 7 decimal digits” for single precision and about 16 for double. The 6-to-9 spread matters when values are printed and read back. Converting an FP32 number to the nearest 8-digit decimal does not always let you recover the original, but 9 decimal digits always do.
Systems break at the edges of the format, so the edges are worth listing:
- Integers: every integer between −16,777,216 and 16,777,216 (±2^24) is exact. Between 2^24 and 2^25, integers round to a multiple of 2, so adding 1 to an FP32 counter at 16,777,216 rounds back to the same value.
- Overflow: values at or beyond 2^128 round to infinity. Under the default round-to-nearest modes, an overflowing result returns ±infinity.
- Underflow: below the smallest normal number, subnormals fill the gap with evenly spaced values instead of flushing to zero. Their precision degrades gradually, down to 1 bit for the smallest non-zero value.
- Invalid results: operations such as 0/0, ∞×0 or sqrt(−1) return NaN, and NaN then propagates through most later operations.
- Exceptions: IEEE 754 defines five of them. Each returns a default value and raises a status flag rather than stopping the computation.
- Ordering: floating-point addition and multiplication are not associative, so the order of operations affects the result.
The last point deserves more weight than it usually gets in GPU work. A parallel reduction can sum the same numbers in a different order, on a different device or with a different kernel. Because order affects the result, it can then return a different FP32 answer. Hardware also differs at the edges. On AMD Instinct MI200 GPUs, the FP16 and BF16 matrix instructions flush denormal inputs and outputs to zero, but the FP32 and FP64 matrix instructions do not.
Why GPUs and deep learning frameworks default to FP32
FP32 remains the most common and foundational floating-point format in GPU computing. Peak GPU floating-point performance is generally quoted as FP32 throughput in TFLOPS. FP32 is also the standard precision for most graphics rendering and general-purpose GPU applications, with wide hardware support and mature optimisation. Binary32 and binary64 are two of the most common floating-point formats in use today. IEEE 754 exists so that a computation gives the same result whether it runs in hardware, software or a mix of the two. Deep learning inherited that default: FP32 has been the workhorse format for most training runs of the deep-learning era.
On a modern NVIDIA GPU, though, “FP32” does not always mean FP32 arithmetic. On Ampere, NVIDIA describes TF32 as the default mode for AI training with 32-bit variables, bringing Tensor Core acceleration without changes to model scripts. In that mode, matrix multiplies and convolutions round their FP32 inputs to TF32, compute the products, and accumulate into an FP32 output. Storage and all other operations stay in FP32.
PyTorch’s documentation describes a different default. In PyTorch, TF32 is disabled for matrix multiplications and enabled for convolutions. Its documentation warns that reading only 10 mantissa bits can produce surprising results, such as a matrix multiplied by the identity not returning the input. The two sources describe different layers rather than contradicting each other: one is the hardware mode NVIDIA’s libraries offer, the other is the framework setting PyTorch ships. Either way, “we ran it in FP32” is an incomplete statement until the TF32 setting is named. NVIDIA also documents a system-wide override: setting NVIDIA_TF32_OVERRIDE=0 disables TF32 kernels so that FP32 kernels are used.
The quoted FP32 peak is a theoretical figure, not a measurement of what a device delivers. A study of the NVIDIA CMP 170HX mining GPU measured FP32 throughput of roughly 0.39 TFLOPS, about 1/32 of its theoretical figure. With fused multiply-add instructions disabled at compile time, it reached about 6.2 TFLOPS, half the theoretical figure. The card is unusual, but it shows that a theoretical FP32 peak is not guaranteed in practice.
Single precision vs double precision: when does a GPU workload need FP64?
Double precision (binary64) uses 1 sign bit, 11 exponent bits and 52 mantissa bits, for a precision of p = 53. That gives about 16 decimal digits against FP32’s 7. Recovering a binary64 value exactly through decimal takes 17 digits.
Workloads that genuinely need FP64 are numerical rather than neural. Goldberg’s classic treatment notes that some algorithms need a short burst of higher precision to produce accurate results. For example, computing b² − 4ac in double precision preserves all the single-precision bits of a quadratic’s roots. Iterative refinement is of little use unless the residual is computed in double precision. A survey of mixed precision algorithms makes the general case: higher precisions cost more but can bring great benefits even when used sparingly. Scientific computing and calculations where error accumulates are where a genuine FP64 requirement shows up.
FP64 hardware support is uneven. In the CMP 170HX study, FP64 ran at about 1/64 of the card’s theoretical peak, falling to 1/128 with FMA disabled. The LynxBenchAI 26Q3 results show the spread across devices. On Dense MatMul in the 26Q3 results, the NVIDIA GeForce RTX 4090 scores 239.9 at fp32 and 9.5 at fp64. In the 26Q3 results, the NVIDIA A100-SXM4-80GB scores 312.4 at fp32 and 132.4 at fp64. If a workload truly needs FP64, that requirement narrows the device choice before any other comparison starts.
FP32, FP64, TF32, BF16 and FP16 side by side
The table shows what each format allocates and what that buys relative to FP32. The wider question of what each floating-point format trades in AI has its own article.
| Format | Sign / exponent / stored mantissa bits | Range versus FP32 | Typical role |
|---|---|---|---|
| FP64 (binary64) | 1 / 11 / 52 | Wider | Scientific computing and high-accuracy subcalculations |
| FP32 (binary32) | 1 / 8 / 23 | Baseline | Default format for GPU compute and most training runs |
| TF32 | 1 / 8 / 10 | Same as FP32 | Tensor Core mode for matmul and convolution; storage stays FP32 |
| BF16 | 1 / 8 / 7 | Same as FP32 | Training without retuning hyperparameters |
| FP16 (binary16) | 1 / 5 / 10 | Narrower | Mixed-precision training with an FP32 weight copy |
Why FP16 diverges in training where BF16 usually does not
Going from 32 to 16 bits did not cut FP16’s capabilities evenly. The exponent budget fell from 8 bits to 5, and because exponent bits govern scale, FP16 loses far more than half of FP32’s dynamic range. Kalamkar et al. found BF16 attractive for training because its range matches FP32 and conversion between the two is simple. Kalamkar et al. report that keeping FP32’s range is important to avoid hyperparameter tuning for convergence, which IEEE FP16 requires. In their study, BF16 training matched FP32 results in the same number of iterations with no hyperparameter changes. BF16 keeps fewer mantissa bits than TF32.
What stays in FP32 inside a mixed-precision pipeline
Even when weights and activations move to 16 bits, FP32 keeps the weight master copy and the accumulations. One mixed-precision training method keeps a single-precision copy of the weights. That copy accumulates the gradients after each optimiser step and is rounded to half precision for training. Half-precision GEMMs usually accumulate in single precision for accuracy and resilience to overflow. PyTorch’s math backend for scaled dot-product attention upcasts FP16 and BF16 inputs to FP32 for the computation, then casts the results back. Attention shows clearly how unevenly numerical sensitivity is spread across a network. Which operations tolerate which format is the subject of how mixed precision exploits numerical tolerance.
What leaving FP32 does to throughput, memory and bandwidth
Fewer bits per value means fewer bytes moved per value. On hardware with matching execution units, it also means more operations per second. NVIDIA reports that moving from TF32 to FP16 mixed precision gives up to about a further 2× speed-up, because 16-bit Tensor Cores are twice as fast as TF32 mode and memory traffic halves. FP8 offers roughly twice BF16’s throughput on supported hardware and halves the memory footprint, so larger models fit on fewer GPUs. Where the gain appears depends on what limits the workload. In LLM inference, prefill is typically compute-bound and decode memory-bound. Weight quantisation therefore mostly speeds up decode by reducing memory movement, while quantising activations as well improves prefill.
That is why a precision choice is never purely about what the model tolerates, nor purely about what the hardware offers. Turning a format change into a lower inference bill is applied engineering work. TechnoLynx covers where the precision lever sits alongside the other levers in performance tuning for AI inference, and how it turns into cost in inference as a production cost.
Changing precision moves numerical representation, not task intent
What a change of precision moves is numerical representation: the range a format covers, the resolution between neighbouring values, the bytes each value costs and the throughput the hardware delivers on it. It does not trade task intent. A classifier moved off FP32 is still meant to compute the same function, and the question is whether the coarser representation still keeps its outputs within the tolerance the task allows. That is why precision is better treated as an explicit design parameter than as a loss of quality. FP32 is one point on that axis, a specific design choice, not the definition of correct behaviour. The longer argument is in precision as a design parameter, not a quality compromise.
How should precision and accuracy be reasoned about as separate concepts in an inference system?
Precision and accuracy answer different questions, and an inference system needs each answered separately. IEEE 754 defines precision as the maximum number of significant digits a format can represent, or the number of digits a result is rounded to. Accuracy is the gap between a result and the real number it approximates, which Goldberg measures in units in the last place or as relative error. Even the floating-point number nearest the correct result can be off by half a unit in the last place.
Higher numerical precision does not automatically mean higher task correctness, for two reasons. First, a wider format still produces one rounded answer whose value depends on the order of operations. An FP32 result can therefore be precise without being guaranteed accurate. Second, task correctness is judged at the output, and tested models reach it with fewer bits. NVIDIA reports that TF32 gave identical accuracy and training behaviour to FP32 across computer vision, language and recommender models. A 2024 study of LLM quantisation found 8-bit floating-point weights and activations essentially lossless against a BF16 baseline. The reverse also holds, which is why accuracy never stops mattering: a format that cannot represent a model’s gradients or activations gives wrong results however fast it runs.
How should precision and recall be kept separate from numerical precision when reasoning about AI system quality?
Precision in “precision and recall” is a classification metric: the share of positive predictions that are actually positive. It has nothing to do with how many bits a number carries. When you reason about AI system quality, keep the two on separate axes. Numerical precision is a property of the system’s arithmetic, while classification precision and recall measure its outputs against labels. The only way to know is to measure the task metric at each numerical precision, not to infer one from the other.
When is reducing precision a sound engineering decision, and when is it not?
The task tolerance is known, the reduced format’s range covers the values the model produces, the hardware executes the format natively, and the task metric has been re-measured at that format. It is not sound when any of those four is assumed rather than checked.
In the 2024 quantisation study, every scheme tested recovered about 99% of the unquantised BF16 baseline across model sizes, and 8-bit schemes averaged 99.75% recovery. The same study also shows the limits. The smallest models at INT4 had larger, though still reasonable, drops. Which 8-bit format applied also depended on GPU generation: integer on Ampere and older, floating point on Hopper and Ada Lovelace.
These conditions argue against reducing precision, or for keeping part of the computation in FP32 or FP64:
- The computation forms residuals from nearby inexact values, where cancellation destroys digits. Iterative refinement is the textbook case.
- The format’s range is narrower than the values in flight, as with FP16 training that has no single-precision weight copy.
- The target device does not support the format, as the A100’s zero fp8 inference scores in the 26Q3 results show.
- The task metric at the new precision has only been inferred from someone else’s model, never measured on the task itself.
For a specific edge deployment, choosing between quantisation, distillation and staying in FP32 is work TechnoLynx takes on in its AI Porting & Deployment Pack. The cost side is argued in when lower precision’s savings are worth the risk.
How does a per-precision correctness threshold change what a speed-up measurement means?
In LynxBenchAI, a result counts only after it passes a correctness threshold set for its precision, so a fast wrong answer scores nothing. Without that gate, a speed-up figure would credit a fast wrong answer. With it, the figure shows how quickly hardware produces acceptable answers. LynxBenchAI evaluates each model across several precisions and reports each one separately: fp64, fp32, fp16 and bf16 on compute workloads, fp32 and bf16 on training, and fp16, int8 and fp8 on inference. Aggregation weights throughput by the memory a run actually moves. A lower-precision run therefore has to earn its speed-up rather than be credited for being cheap.
Here is how that reads on a device page. In the 26Q3 results, the NVIDIA H100 80GB HBM3 scores 1026.7 at fp32, 1264.3 at fp16, 1315.1 at bf16 and 414.4 at fp64 on Dense MatMul on the LynxBenchAI 26Q3 Index. Each figure is a separate measurement of the same workload at a different precision, and each non-zero score has cleared its own precision’s threshold. The training figures show that a lower precision does not automatically score higher. On DistilGPT2 training, the same device scores 1159.0 at fp32 and 651.9 at bf16.
The measurement has declared limits. Each test is one timed window, not a median across repeated trials. The scores cover the fixed model catalogue of one named release, not the reader’s own application.
What happens in a benchmark result when a device cannot execute a given precision?
When a device cannot execute a precision, LynxBenchAI scores that precision as zero rather than quietly skipping it, so the gap stays visible in the result. In the 26Q3 results, the NVIDIA A100-SXM4-80GB scores 1645.7 at fp16, 759.3 at int8 and 0.0 at fp8 on Flan-T5 Small inference. It also scores 0.0 at fp8 on every other inference model in the release. That zero is information. If fp8 were skipped, the device’s inference picture would look complete even though one of its precisions produced nothing, and any summary built on it would quietly average over fewer tests.
Measuring each precision on your own hardware
The LynxBenchAI Personal Edition shows how your device handles each precision, gated by the same correctness thresholds; run it with pip install lynxbench-ai. It runs the release’s fixed catalogue on your hardware, so it does not show how your own model will behave at those precisions.
What remains for you to settle is the tolerance your task actually has. Once you know it, the bit width stops being a matter of habit and becomes a design choice you can check on your own task.
