The verified evidence stops at about 3B to 3.9B parameters: BitNet b1.58 matches half-precision baselines from 3B, and the independent Spectra suite shows a 3.9B ternary model matching a 3.9B float model.
LynxBenchAI publishes measured hardware performance results for the accelerators a ternary-versus-FP16 ablation runs on, and the first step is to check its published results for your target stack before the ablation is sized.
TechnoLynx designs and runs the small matched-data, matched-compute ablation that tells you whether ternary tracks FP16 on your data and your accelerator stack, before the full cluster budget is committed. The first step is to fix the protocol: data mixture, token budgets and the model sizes the scaling fit needs.
Has anyone published ternary pretraining results at 30B parameters?
Not in the papers cited here: their 30B model is the 1-bit BitNet b1, and Spectra’s ternary scaling study stops at 3.9B. Among the papers reviewed, the matched quality comparisons stop at the BitNet b1.58 runs at 3B and 3.9B and the Spectra TriLM suite, which tops out at 3.9B. The b1.58 paper’s configuration table lists sizes of 700M, 1.3B, 3B and 3.9B. For a fair comparison those models were all pretrained on RedPajama for 100 billion tokens.
Microsoft’s BitNet b1.58 2B4T report (2025) describes a 2B-parameter model trained on 4 trillion tokens that performs on par with leading open-weight full-precision models of similar size.
One caveat on the absence: an unpublished 30B+ ternary run may exist. What does not exist is a published one you can inspect, and a budget decision cannot rest on a run nobody can show you.
The 30B BitNet result is a 1-bit model, not BitNet b1.58
The BitNet work proposes both BitNet b1 (1-bit weights) and BitNet b1.58 (ternary weights). The 30B model belongs to b1: the authors trained a series of BitNet b1 language models from 125M to 30B, as the original 2023 BitNet paper by Wang et al. also reports.
The authors fitted a power law with an irreducible loss term, L(N) = aN^b + c, on the 125M to 6.7B models and used it to predict the 13B and 30B losses, reporting that the fit predicted b1’s loss with high accuracy. Downstream zero-shot and few-shot accuracy, however, was reported while scaling from 1.3B to 6.7B, and the detailed benchmark comparison was run at 6.7B.
For the 13B and 30B models the authors set weight decay to 0.05 for training stability.
What the ternary papers actually measured
The table separates what was trained and evaluated from what was extrapolated or measured for cost only.
| Source | Precision | Trained and evaluated | Extrapolated or cost-only |
|---|---|---|---|
| BitNet b1 (Wang et al., 2023) | 1-bit | Downstream evals to 6.7B; loss fitted on 125M to 6.7B and predicted for 13B and 30B | Trained 30B model, 1-bit rather than ternary |
| BitNet b1.58 | Ternary | Parity with FP16 from 3B; 3.9B tested, 100B tokens | Latency and memory at 7B, 13B and 70B, evaluated for cost |
| Spectra (Nolano, 2024) | Ternary | TriLM 3.9B matches FloatLM 3.9B, 300B tokens | None; scaling laws studied only to 3.9B |
| Spectra-1.1 / TriTera (ACL 2025) | Ternary | Scaling-law runs to 1.1B; 1.5B, 2.5B and 3.6B on 1.2T tokens | Kernel benchmarks on layer shapes from 3B to 405B models |
| BitNet b1.58 2B4T (2025) | Ternary | 2B on 4T tokens, compared with open models of similar size | None |
The two strongest ternary sources do not say quite the same thing at their top end. BitNet b1.58 reports the gap to FP16 shrinking as the model grows, with the 3.9B ternary model outperforming the 3B LLaMA baseline. Nolano’s own summary of Spectra says that up to 3.9B “FloatLMs are clearly a better choice”, while the difference narrows considerably above a billion parameters and the trend shows potential for TriLMs to match or exceed FloatLMs of the same parameter count. Both describe a closing gap, and neither gives a measured quality result beyond about 4B. Spectra released 54 models from 99M to 3.9B.
The 7B, 13B, 70B and 405B figures measure cost, not quality
Large model sizes do appear in the ternary literature, but as cost measurements. The b1.58 authors scaled the model size to 7B, 13B and 70B and evaluated the cost: latency and memory trends, with the 70B model 4.1 times faster than the LLaMA baseline. Those are efficiency measurements.
Statements such as “30B BitNet b1.58 is more efficient than 7B FP16 LLM” in latency, memory and energy are read off the Figure 8 cost trends.
The kernel results go higher again. Spectra-1.1 benchmarked its TriRun kernels on matrices from models of 3B to 405B parameters, reporting roughly a 7.98x speedup over FP16 on an NVIDIA L40 for the 405B shapes at batch sizes of 16 to 32. That is a kernel-level benchmark of the ternary layers. It shows that ternary kernels can pay off at large shapes; it does not show that a ternary model of that size can be pretrained to parity. The Spectra-1.1 authors say plainly that their pretraining scale was constrained by compute and that both parameters and data need to be scaled up significantly to make TriLMs competitive with current state-of-the-art models.
Why should a 30B ternary plan start with small ablations rather than a full run?
The precision-aware slopes you would be betting on were fitted on small models. The general precision-aware scaling laws from Kumar et al. (2024) were fitted on over 465 pretraining runs and validated on models up to 1.7B parameters trained on up to 26B tokens. The ternary-specific laws in Spectra-1.1 come from parameter sizes up to 1.1B (excluding embeddings). The fits are good inside that range, with R² of 0.9921 for TriLMs and 0.9958 for FloatLMs, but a good fit at 1B is a statement about 1B. Extrapolating a power law by more than an order of magnitude is exactly where small exponent errors compound.
Spectra-1.1 found that adding training tokens lowers TriLM validation loss more effectively than adding parameters, so TriLMs stay effective at high token-to-parameter ratios. A 30B plan that copies a float recipe’s token budget may therefore be testing the wrong configuration, and an ablation that varies only model size will miss the axis ternary responds to most.
Three more differences between a 3B experiment and a 30B run are worth checking before you commit:
- Recipe. The BitNet b1 authors set weight decay to 0.05 for the 13B and 30B models for training stability.
- Hardware, kernels and optimiser. Ternary kernel speedups vary with GPU, batch size and model size, so re-verify them on the target stack.
- Evaluation. Parity on commonsense and knowledge benchmarks does not guarantee parity on the tasks you care about; the TriTera benchmark table, for example, shows low GSM8K exact-match scores of 2.05 to 9.70.
A matched-data ablation protocol for ternary versus FP16
This is the order we would run it in, so that each step’s output feeds the next:
- Fix the data. Use one data mixture and tokenizer for both precisions, at the composition the full run will use, as the b1.58 authors used RedPajama for a fair comparison.
- Choose a size ladder. Train at least four or five sizes per precision across an order of magnitude or more, mirroring how the BitNet fit used 125M to 6.7B; more points make the exponent estimate more trustworthy.
- Vary tokens as well as parameters. Run each size at several token budgets, as Spectra-1.1 did with 20B to 150B tokens, so the fit captures the token dependence, which lowers TriLM loss more than added parameters.
- Compare at matched compute. Lay the runs out on IsoFLOP curves so ternary and FP16 are compared at equal training compute, not only equal parameter count.
- Fit both precisions separately. Fit a loss law in parameters and tokens for each, in the parametric form Spectra-1.1 used, and compare exponents and irreducible terms rather than single loss points.
- Evaluate downstream on your own suite. Run the benchmarks your product depends on at every size, not only perplexity.
- Hold out a check point. Keep the largest ablation sizes out of the fit and test whether it predicts them, as the BitNet authors did by fitting to 6.7B and predicting 13B and 30B.
- Set go/no-go criteria before the results arrive. Agree in advance how large a projected loss or benchmark gap at 30B is acceptable, and what fit uncertainty makes the projection unusable.
If the full run goes ahead, keep the fitted curve as a live reference: log loss against the predicted trajectory and treat early divergence as a stop-and-diagnose signal, not something to train through. Cluster sizing for that run should follow profiling of the ternary training step on the actual stack, not throughput assumptions carried over from FP16. Checkpoint tooling for ternary weights is covered in auditing ternary weight compatibility with existing checkpoint tooling.
Who can run a ternary-versus-FP16 ablation before we commit a 30B budget?
TechnoLynx frames and runs the ablation that decides whether a 30B ternary run is worth starting. TechnoLynx sets up the matched-data, matched-compute comparison on your accelerator stack and delivers loss curves, downstream comparisons and fitted slopes for both precisions. The engagement ends with a go/no-go recommendation and monitoring hooks for divergence during the full run. How we keep large training runs and deployed models dependable once they leave the lab is covered under production AI reliability. On internal ternary runs at 10B+: TechnoLynx has no published case study at that scale and would not present small-scale results as one.
This does not suit a team that has already committed to the full run and wants it executed as specified, or one whose question is purely which hardware to buy; for that, the published LynxBenchAI results are the place to start, as they measure hardware performance. Waiting for a lab to publish a 30B ternary run is also a legitimate route if the timeline allows it.
The question your ablation has to answer
The public record supports a narrower claim than the 30B headlines: BitNet b1.58 reports FP16 parity from 3B, and both it and Spectra see the gap narrowing with size. Whether it keeps shrinking, flattens or reopens past 10B is not known, and it is the one fact a 30B budget depends on. A small ablation will not settle that with certainty either. What the ablation delivers is loss curves, downstream comparisons and fitted slopes for both precisions, measured on your data and your stack.
