A prediction you can plan around names three things: the unit it measures, the horizon it covers, and the evidence that would retire it. Anything missing one of the three is commentary — sometimes interesting commentary, but not planning input. That single filter does most of the work when you sit down with a stack of forecasts about where AI is going.
The reason this matters is practical rather than philosophical. Undated capability claims have a way of ending up in roadmap decks, and once they are there, they start re-scoping architecture work that had perfectly good reasons to exist. We see teams defer a retrieval redesign because “context windows will make retrieval unnecessary soon” — a sentence with no unit, no horizon, and no way to be wrong. The redesign still needed doing.
How should a reader evaluate predictions for AI in the future?
Three sequential tests separate credible forecasts from speculation.
Unit. What is being measured, in what currency? “Models will get much cheaper” fails. “Cost per million output tokens for a frontier-class model” passes, because you can look it up today and look it up again later. Units that survive contact with a finance system — cost per request, latency at the 95th percentile, share of workloads running on-premise, time-to-deploy for a new model version — are the ones worth carrying forward.
Horizon. By when, stated as a date or a bounded range. A forecast without a horizon cannot be scored, and a forecast that can never be scored will never be retired; it just accumulates. Note that a year attached to a headline (“AI in 2030”) is not automatically a horizon. It is a horizon only if the claim it dates has a unit.
Falsifier. What observation would make you drop this? If you cannot answer, you are not holding a forecast, you are holding a mood. This is the test most predictions fail, and it is the cheapest one to apply.
A prediction that passes all three earns a place in planning. A prediction that passes one or two becomes a watch item with a review date. One that passes none gets excluded rather than hedged against — hedging against unfalsifiable claims is how architectures acquire options nobody ever exercises.
Four categories of prediction, and why mixing them misleads
The most common reading error is treating all forecasts as the same kind of object. They are not. Compute-cost curves, deployment patterns, capability jumps, and regulatory posture have different evidence bases, different error profiles, and different consequences for a roadmap. Conflating them is what produces the “everything changes next year” reading that survives no audit.
| Category | Typical unit | Evidence base | Error profile | Planning weight |
|---|---|---|---|---|
| Compute cost | Cost per token, per request, per training run | Published price lists, historical trend, hardware roadmaps | Direction usually right, timing often early | High — maps directly to a budget line |
| Deployment pattern | Share of workloads on-premise vs hosted; adoption percentage | Vendor surveys, analyst reports (methodology varies) | Sensitive to sample and definition; adoption is often over-counted | Medium — useful for capacity and hiring |
| Capability jump | Benchmark score, task success rate, “human-level at X” | Leaderboards, model releases, expert opinion | Widest error bars; thresholds get redefined mid-flight | Low — watch item unless a specific benchmark and date are named |
| Regulatory posture | Enacted obligations, compliance deadlines | Draft legislation, published timetables | Timing slips, but text is inspectable | High when a deadline exists; low for “regulators may act” |
Two observations from this table are worth stating plainly. First, compute-cost and regulatory forecasts are the two categories where a stated date usually comes attached to an inspectable artifact — a price list, a legislative timetable — which is why they carry more planning weight than capability forecasts even when the capability forecast sounds more consequential. Second, capability forecasts are the ones most likely to move a roadmap and least likely to survive scoring, which is an unpleasant combination and the reason we default them to watch items.
Benchmark-anchored capability claims deserve a further caution: the measurement instrument is itself contested. A forecast pinned to a leaderboard position inherits every weakness of that leaderboard’s methodology, which is why how the Chatbot Arena Elo ranking is actually constructed matters before you plan around a predicted rank. Similarly, hardware forecasts read off headline specifications need the sustained-versus-peak correction covered in our piece on how hardware specs shape AI infrastructure performance — a predicted throughput figure derived from a spec sheet is a prediction about a number, not about a system.
Named scenario documents and dated years
Named forecast programmes circulate as planning input — scenario write-ups with a year in the title, published timelines from research groups, in-house strategy documents that quote them. The frame does not change for these. Open the document and ask which of its claims carry a unit, which carry a horizon, and which name a falsifier. Well-built scenarios usually do declare their assumptions and branch points, and those branch points are exactly the falsifiers you need; the failure is on the reading side, when a scenario written as if these conditions hold, then gets quoted as this will happen by.
When a prediction is stated as a year rather than as a condition, the missing falsifier tells you something specific: the author is describing an expectation, not a forecast you can score. “AI in 2030” as a headline is a rhetorical container. The planning-relevant question is what would have to be observed in 2027 for the 2030 statement to still be live — if nobody can name that, the claim carries no weight beyond narrative.
Labour-displacement forecasts sit in the same bucket for most readers. “Which jobs survive AI” is a real question with real stakes, and it is also almost always stated without a unit you can measure inside your own organisation. Under this frame it is a watch item, not a procurement input — with one exception: if a specific role in your own hiring plan has a task profile you can decompose and measure automation coverage against, that decomposition gives you a unit and a horizon, and you are no longer reading someone else’s forecast at all.
What generative models can and cannot tell you here
A related confusion is worth separating out, because it changes what tooling you reach for. Predictive forecasting methods — time-series models, regression on historical cost and adoption data, scenario simulation — produce estimates with error bars derived from data. A large language model asked “what will AI look like in five years” produces fluent synthesis of published opinion, with no error bars and no distinction between a well-evidenced trend and a widely repeated one. Both outputs are useful; only one of them is a measurement. Treating generated prose as a forecast is the fastest way to import unfalsifiable claims into a roadmap, because the output arrives pre-formatted to look like analysis.
In our experience, the useful move is to use the model for what it is good at — surfacing the range of positions on a question, naming the assumptions each position rests on — and then do the unit/horizon/falsifier sorting yourself. That sorting is not automatable, because it depends on which metrics your organisation actually tracks.
A working checklist
Use this on any forecast before it enters a planning document:
- Name the unit. If you cannot state it in the currency of a metric you already track, stop here.
- Name the horizon. A date or a bounded range, not “soon” and not a headline year.
- Name the falsifier. One observation that would make you drop it. Write it down next to the prediction.
- Classify the category. Compute cost, deployment pattern, capability, or regulation. Do not average across categories.
- State the decision it touches. A budget line, an architecture choice, or a hiring plan — one of the three, named.
- Set a review date. Retained predictions get re-scored; watch items get re-read. Both get a date.
- Exclude, don’t hedge. If steps 1–5 fail, remove it from planning rather than building an option against it.
The output of this exercise is short, and that is the point. Most teams we work with end up with three or four forecasts that survive and a much longer list parked as watch items with review dates — which is a far more honest input to a roadmap than a deck of headline numbers. The broader question of how to read trend material without letting it drive decisions is where our overview of AI trends and predictions goes further, and the general framing of how we approach this work sits on our main practice page.
One uncertainty is worth naming rather than papering over. This frame is deliberately conservative about capability forecasts, and a conservative frame will occasionally be late — if a capability threshold is genuinely crossed early, the teams that hedged will look prescient. We accept that cost, because the alternative is a roadmap re-scoped every quarter by whichever claim was loudest. What we have not solved is how to price the option value of hedging against a low-probability capability jump; if you have a method for that which does not collapse into guessing, it would change step 7.
Frequently Asked Questions
How should a reader evaluate ‘predictions for ai in the future’? Apply three tests in order: does the prediction name a measurable unit, a dated or bounded horizon, and an observation that would retire it? A forecast passing all three can enter planning; one passing some becomes a watch item with a review date; one passing none should be excluded rather than hedged against.
What separates a testable AI forecast from undated commentary? The falsifier. A testable forecast tells you what you would have to observe to drop it, which means it can be scored later and eventually retired. Commentary states a direction without a scoring condition, so it never expires and quietly accumulates in strategy documents.
Which categories of AI prediction — compute cost, deployment pattern, capability, regulation — behave differently, and why does conflating them mislead? Each has a different evidence base and error profile: cost forecasts usually get direction right and timing early, adoption figures are sensitive to survey definitions, capability claims carry the widest error bars, and regulatory claims are inspectable when a deadline exists. Conflating them averages a well-evidenced price trend with a speculative capability threshold, producing a confidence level neither claim earns on its own.
How far out can AI predictions be treated as planning input rather than watch items? The limit is set by the category rather than by a fixed number of years. Where a claim maps to an inspectable artifact — a published price list, a legislative timetable — it can carry planning weight to that artifact’s own horizon; capability claims generally stay watch items regardless of how near-term they sound.
What evidence would retire a prediction you are currently planning around? That is a question you should be able to answer for each retained forecast, written down beside it. For a cost forecast it is typically a published price that fails to move by the stated date; for a deployment-pattern forecast, an internal workload census that contradicts the projected split.
How do predictive AI forecasting methods differ from what generative models can tell you about the future? Predictive methods fit historical data and produce estimates with error bars you can inspect. A language model produces fluent synthesis of published opinion without error bars and without distinguishing a well-evidenced trend from a widely repeated one — useful for surveying positions, not a measurement.
Which future-of-AI predictions actually change an architecture or procurement decision this year? The ones that pass the three tests and name a decision they touch: a budget line, an architecture choice, or a hiring plan. In practice that shortlist is dominated by compute-cost and regulatory-deadline claims, because those are the two categories where a date arrives attached to something you can read.
Named forecast programmes such as the ‘AI 2027’ scenario or the AI Futures Project circulate as planning input — how should a reader test one of these dated scenarios against the unit/horizon/falsifier criteria? Open the document and sort its individual claims rather than accepting the headline. Well-constructed scenarios declare assumptions and branch points, and those branch points are the falsifiers; the reading error is quoting a conditional scenario as an unconditional timeline.
When a prediction is stated as a year (‘AI in 2030’, ‘AI in 2050’) rather than as a condition, what does the missing falsifier tell you about how much planning weight it can carry? It tells you the author is expressing an expectation, not issuing a scorable forecast. The planning-relevant reformulation is to ask what would have to be observed well before that year for the claim to still be live — if nobody can name it, the claim carries narrative weight only.
How should a reader handle the widely circulated labour-displacement forecasts (‘which jobs survive AI’) — are they procurement-relevant predictions or watch items under this evaluation frame? Watch items, in almost every case, because they rarely come with a unit you can measure inside your own organisation. They become planning input only when you decompose a specific role in your own hiring plan into tasks and measure automation coverage against it — at which point you have built your own forecast rather than adopted someone else’s.
Three filters for credible AI forecasts
Discard any prediction that doesn’t specify falsifiable conditions, concrete timelines, and acknowledged uncertainty bounds. If not, it’s marketing. Everything else is detail.
