A ternary BitNet b1.58 pilot is worth scaling only if it holds up against two other runs: an INT4 quantisation-aware training (QAT) run and a bf16 run, both with the same parameter count, tokens and compute, and judged on downstream benchmarks rather than on loss. From about 3B parameters, published matched comparisons report ternary parity. The BitNet authors (JMLR, 2025) report that b1.58 matches half-precision baselines from 3B at the same model size and training tokens, and Spectra’s 3.9B ternary model matched its float counterpart trained on the same 300B tokens. Researchers still dispute that verdict.
LynxBenchAI benchmarks throughput, latency, memory and energy across accelerators and precisions, so a team choosing hardware for the ternary, INT4 QAT and bf16 arms starts by measuring its candidate accelerators with it.
TechnoLynx runs this comparison as a matched-budget pilot. It starts by framing the pilot as a controlled experiment. It then fixes parameters, tokens and evaluation suite across ternary, INT4 QAT and bf16 arms on the target accelerators, and ends with a report of the accuracy and stability deltas and a recommendation to scale or stop.
What has to be held equal across the ternary, INT4 QAT and bf16 arms
A fair comparison holds parameter count, training tokens (with the same data ordering), training compute, model shape and quantised scope equal. On top of that, the learning rate is tuned separately for each arm, not copied across. Spectra shows the data side in practice. All of its ternary and float models were trained on identical data sequences from the same 300B-token SlimPajama subset, with data ordering and batch sizes kept consistent within each model family.
In quantisation-aware training of weights only, the matrix multiplications still run in high precision, so the quantised weights save no compute during training. So a weight-only ternary or INT4 QAT arm saves no training compute over a bf16 arm at the same parameters and tokens. That only holds if the arms are otherwise identical. The picture changes for low-precision training, where quantising activations too allows compute gains, and Kumar et al. (ICLR 2025) model training compute as scaling linearly with precision.
Memory matching asks a different question. The 1-Bit Wonder study (2026) paired a 4B bf16 model trained on 150B tokens with 4-bit and 1-bit counterparts of roughly 12B and 31B parameters, all at the same inference weight memory. That design tests which format gives the best model under a fixed inference memory budget. It does not tell you whether ternary loses accuracy at the parameter count you already train.
Matching axes and what each one answers
| Hold equal | What goes wrong if you don’t | Question it answers |
|---|---|---|
| Parameter count and tokens | Scale gets confused with budget: one published comparison gave one model 650× more data, which would produce a “reversal” with no size effect at all | Does ternary lose accuracy at our model size? |
| Training compute | Only differs from the row above when activations are quantised too, since weight-only QAT saves no compute | Which precision is compute-optimal? |
| Inference memory | Scaled-up low-bit models beat the 4B bf16 baseline on every benchmark, which is a different finding from parity at equal size | What is the best model for a fixed memory budget? |
| Model shape | Transformers matched on parameter count varied by 22.6% in loss purely through depth and width choice in a 2026 controlled re-run | Is the baseline the strongest one of its size? |
| Learning rate per arm | An untuned from-scratch ternary baseline may inflate every ternary penalty | Is the gap about precision or about tuning? |
| Quantised scope | Arms that keep different shares of parameters in full precision are not equally quantised | Are the arms actually comparable? |
On scope, ParetoQ quantises all weights except the embedding and output layers, following BitNet b1.58. Spectra likewise left embeddings, the language-model head and activations unquantised in its INT4 comparison. Whichever choice you make, apply it to every arm.
BitNet b1.58 vs bf16 comparison at matched parameters and tokens
Matched comparisons support parity from 3B to 3.9B parameters. Below that, Spectra found a gap at 1.1B. The original BitNet b1.58 report (Ma et al., 2024) says the ternary model starts to match full-precision LLaMA perplexity at 3B, and that the 3.9B ternary model outperforms the 3B LLaMA baseline on end tasks. Those models were pretrained on 100B tokens of RedPajama for the comparison.
Spectra (Kaushal et al., 2024) trained its own float and ternary families on identical data. At 3.9B, its ternary model matched the float model across benchmarks despite higher perplexity. At 1.1B and 100B tokens, it did not come close to its float counterpart. The authors also report that their 3.9B ternary model does worse on datasets that may overlap with the training data while staying competitive out of distribution.
It was trained on 4 trillion tokens and compared against instruction-tuned open models of similar size. In its table against Qwen2.5 1.5B in bf16, the ternary model averages 55.01 across MMLU, GSM8K and IFEval, against 55.72 for bf16, and trails on MMLU (53.17 against 60.25). On a smaller budget, Nielsen et al. (2025) found 16-bit models scored higher on average than every 1.58-bit configuration across 11 downstream tasks, by small margins of unclear significance.
How large is the accuracy gap between ternary and INT4 QAT at the same scale?
A pilot should produce its own ternary-versus-INT4 QAT number at matched tokens below 3B.
ParetoQ (2025) compares bit-widths in one framework, with QAT applied as fine-tuning after full-precision training. Its authors find that ternary, 2-bit and 3-bit quantisation tie on the trade-off between model size and accuracy, and often beat 4-bit. That is a comparison on the size-accuracy trade-off. They also report that 3-bit and 4-bit networks recover near full-precision accuracy after about 10B fine-tuning tokens, while ternary saturates around 30B tokens before reaching full accuracy. Their ternary 3B model sits 3.3 points below a full-precision LLaMA-3 3B at 65.2. So in ParetoQ’s results, 4-bit recovers closer to full precision than ternary does.
Spectra compares against INT4, but with post-training GPTQ rather than QAT. At the largest scales, its ternary models beat the 4-bit models with the same parameter count. The 2B4T report likewise beats GPTQ and AWQ INT4 versions of Qwen2.5 1.5B, but again those are post-training baselines. Neither compares ternary with an INT4 QAT arm.
A 2023 study of INT4 transformers found that quantising both weights and activations to 4 bits (W4A4) caused a significant accuracy drop for decoder-only models, even with QAT, distillation and extensive tuning. Quantising only the weights to 4 bits almost matched the teacher’s quality. So for decoder models, a W4A4 QAT arm is a weaker baseline than a weight-only one, and you should state which you ran.
Recipe changes ternary training needs that an INT4 QAT pipeline does not
Ternary training needs more than a different quantiser in your INT4 QAT pipeline. It changes the quantiser, the learning-rate schedule and the weight-decay schedule. Kumar et al. note that BitNet uses a nonstandard architecture, learning rate and weight-decay schedule specifically to make training with ternary weights stable.
| Weights during training | Native bf16 | Group-wise 4-bit quantisation; finer groups reduce approximation error | High-precision latent weights quantised on the fly, gradients through a straight-through estimator |
| Weight quantiser | None | Grids that include 0 work better at 3 and 4 bits; learnable ranges beat statistics-based ones | Absmean scaling to {-1, 0, +1}; absmedian is an alternative worth treating as a hyperparameter |
| Activations | bf16 | Often weight-only, since decoder models are very sensitive to activation quantisation | 8-bit, absmax per token |
| Normalisation | Standard pre-norm | - | SubLN chosen for stability |
| Learning-rate schedule | Single cosine or linear decay | ParetoQ’s QAT phase uses 1×10⁻⁴ with no weight decay | Two stages: high peak, then an abrupt cooldown around the midpoint |
| Weight decay | Constant regulariser | As above | Removed for the second half of training, because it acts on latent weights and makes ternary weights flip more often |
| Precision over the run | Fixed | About 10% QAT after full-precision training | 16-bit for the first 20 to 40% of the data, then 1.58-bit |
Two of these deserve care in a pilot. The first is the full-precision-then-ternary schedule. ParetoQ finds accuracy peaks at roughly 90% full-precision training and 10% QAT. The 2026 controlled re-run found that a 90/10 schedule beat all-ternary training only with a stage-2 learning rate about 10× the pretraining peak, and looked 15.3% worse at a conventional fine-tuning rate. Ship the schedule without its stage-2 learning rate and the conclusion reverses.
The second is the direction of the learning rate, and here the sources disagree. The 2B4T report uses a high initial peak because 1-bit models tend to train more stably. A 2024 study of models from 100K to 48M parameters found that larger learning rates did not help 1.58-bit small language and vision models, and in its sweep the smallest rate did best. We give more weight to the small-model study at pilot scale, because it measured that regime directly, while the high-rate guidance comes from LLM runs. The safe course is to sweep the learning rate per arm rather than inherit either rule.
Gradual quantisation is a third, smaller choice. Hugging Face’s write-up on fine-tuning to 1.58 bits introduces quantisation through a lambda ramp. Nielsen et al. found that gradually phasing in quantisation strength gave little to no benefit once training had enough steps to catch up. The two findings are compatible: a ramp eases the transition, but a long enough run absorbs the difference.
Loss spikes and gradient problems in small ternary runs
Ternary pretraining does produce loss spikes. How bad they are depends on scale and recipe, so you have to look for them in the curves rather than infer them from the final loss. In Spectra, TriLM 2.4B and 3.9B each had a large loss spike in the first half of training. The same learning-rate cut at the midpoint that helped the larger models made the loss plateau at 1.1B and 1.5B, and made it rise for models below 1B parameters. So the same recipe change stalled Spectra’s smaller models but not its 2.4B and 3.9B ones.
The curve shape also misleads early readings. The BitNet authors observe that 1-bit training often follows an S-shaped loss curve with a sharp drop near the end, so intermediate evaluations may not predict final performance. Stopping a ternary pilot early because it trails bf16 at the halfway mark is a decision made on the least reliable part of the curve.
The sources document these mitigations:
- Retaining optimiser state across the 16-to-1.58-bit switch reduces the size of the transition loss spike, though a fresh optimiser reaches similar loss after a limited number of steps.
- Subtracting the tensor mean before quantisation was how the 1-Bit Wonder authors mitigated the NaN-producing instabilities they saw in uniform 1-bit training.
- Removing weight decay at about two-thirds of training sped up convergence for every Spectra model.
- Continued pre-training as a warm-up countered the instability and widening gap that BitDistill (Microsoft, 2025) reports when QAT is applied directly to pretrained models at 1.58 bits.
For instrumentation, we would log loss and gradient norms for every arm from the first step. We would read spikes against the shared data ordering, because Spectra’s minor spikes appeared at the same token counts across all scales, which Spectra ties to the shared data ordering.
Which evaluation suite catches ternary-specific regressions
Pretraining loss and log-likelihood evaluations are not enough to judge a ternary arm. Downstream generative tasks are the part of the suite that decides. The 1-Bit Wonder authors found that logprob evaluations during pretraining put their 4-bit model on par with or ahead of the 1-bit model, while downstream generation showed the opposite. They also note that several influential ternary papers rely mainly on loss or token-level likelihood. In the wider compression literature, Jaiswal et al. (ICLR 2024) showed that perplexity fails to capture subtle changes in the true capabilities of compressed models.
A diagnostic checklist for the shared suite
- Run every arm through EleutherAI’s lm-evaluation-harness, whose public prompts are there for reproducibility and comparability. Spectra used the same harness for its ternary, float and quantised comparisons.
- Cover the commonsense set the BitNet papers report: ARC Challenge, ARC Easy, BoolQ, HellaSwag, PIQA and WinoGrande.
- Add knowledge, math and instruction following: MMLU, GSM8K and IFEval are where the 2B4T comparison showed the largest swings.
- Add code generation (HumanEval, MBPP), where the memory-matched 1-bit model’s advantage appeared.
- Split in-distribution from out-of-distribution scores, since ternary models can trail on the first while matching on the second.
- Include bias or toxicity benchmarks: Spectra’s sub-1B ternary models stereotyped less than float models of the same size, and the difference disappeared at larger scales.
- If the model will be instruction-tuned or used with tools, add reasoning retention and tool-call validity, which a 2025 survey of low-precision training says future protocols should add to the loss, accuracy, memory and throughput most studies report.
Agree the pass threshold for each benchmark before the runs finish. The threshold is a decision about operational consequence, not something the benchmark produces, and setting it after seeing the results invites a fight over who moved the goalposts.
How small can a pilot ablation be and still tell you something?
A pilot well below 3B parameters shows how the gap trends with size, as the small-model study did. It cannot tell you whether ternary reaches parity, because the parity claims start at 3B. The 2024 small-model study found that 1.58-bit networks needed roughly double the hidden size to match 16-bit performance. Spectra saw no parity at 1.1B. A 300M-parameter pilot that shows a gap to bf16 has therefore reproduced the literature, not ruled ternary out.
We would run a ladder of sizes rather than one point, with more than one seed per arm. A 2026 re-examination found the sub-1M-parameter regime supported mainly by isolated, single-seed comparisons, and that the ordering of baseline shapes reversed with budget. Scaling-law extrapolation also rests on a trend fitted across sizes. Kumar et al. fitted their precision laws on 30M to 220M-parameter runs and validated them up to 1.7B parameters and 26B tokens, and the BitNet authors fitted 125M to 6.7B models to predict 13B and 30B loss. Even so, Spectra’s recipe effects changed with model size. Spectra’s ternary-float comparison stops at 3.9B parameters, so parity at 30B is a separate question. Treat it as open until someone measures it at that size.
Who can run a matched-budget ternary pilot on our cluster?
TechnoLynx designs and runs the controlled comparison for teams deciding between ternary, INT4 QAT and bf16 training. The work starts by framing a controlled experiment that fixes parameters, tokens and evaluation suite across the three arms. Next comes a consistent implementation of the ternary recipe (straight-through estimator variant, learning-rate schedule) and the INT4 QAT pipeline on your accelerators. The small-scale runs are instrumented for loss spikes and gradient norms and validated against bf16 on the shared suite, and the result is a report of the deltas with a recommendation on whether to scale. The engagement sits under Generative & Agentic AI R&D. If the decision also involves choosing accelerators, LynxBenchAI benchmarks throughput, latency, memory and energy across accelerators and precisions. It measures hardware, not your training run.
If the goal is a smaller model for a fixed task rather than a pretraining decision, a distillation route like BitDistill, which fine-tunes existing models to 1.58 bits, needs only a small fraction of the training of pretraining from scratch. We also cannot promise that pilot-scale results transfer to your full target size; nobody can.
Scale, hold or stop: reading the pilot
| Pilot result at matched budget | Decision |
|---|---|
| Ternary matches or beats INT4 QAT on downstream tasks, with no instability INT4 lacks, and the gap to bf16 narrows across the size ladder | Run a larger bf16-versus-ternary comparison before committing the full cluster budget |
| Ternary trails INT4 QAT but the gap shrinks with size, and the learning-rate sweep was thin | Hold: tune per arm and add a size before deciding, since untuned baselines can flip the sign |
| Ternary shows accuracy collapse or instability that INT4 QAT does not, after per-arm tuning | Keep INT4 QAT or bf16 as the safer path |
| Ternary wins only on loss or logprob evals | No decision yet: those metrics did not predict downstream generation in memory-matched tests |
A win on accuracy is still not a win on serving cost. ParetoQ’s authors point out that ternary values are inefficient to store and complicate GEMM, making 2-bit generally more feasible. Low-bit weight kernels without hardware support lose their speedup as batch size grows, and in the 1-Bit Wonder measurements it is gone by batch 256. Before scaling, check that the format can be served at your production batch sizes and on your kernels, as well as trained.
FAQ
Is bfloat16 better than float16 for training?
BF16 matches the dynamic range of FP32, which removes the need for hyperparameter tuning, and the 2025 low-precision training survey reports it has gradually replaced FP16 as the default 16-bit format. That range advantage is the core of the fp16 vs bf16 choice.
What is the difference between post-training quantisation and quantisation-aware training?
Quantisation-aware training builds quantisation into training, often through fake-quantisation operators, so the model adapts to low-precision arithmetic before deployment. Post-training quantisation quantises a model after training, before inference. Modern PTQ methods stay competitive down to about 4-bit weights, but they degrade quickly below that.
