Generative AI for business has its clearest route to measurable return in repeatable, menial tasks that current models can already do. Tasks that quietly need better-than-human capability are a fundamentally different proposition. The published evidence on returns is sharply split. In 2025 a Google Cloud study found 74% of enterprises seeing ROI within the first year, while MIT’s NANDA initiative found 95% of pilots delivering no measurable P&L impact, and CloudZero’s reading is that the gap comes from whether a company can measure cost and outcome per use case, not from the models. For the productivity side of the same question, our overview of generative AI in analytics and business workflows covers where analysts and knowledge workers gain.
Generative AI for business starts with a feasibility test
Before anyone talks about cost or vendors, separate the tasks into three piles. Our marketing feasibility piece observes that most marketing teams adopting generative AI mix tasks current models can already automate, tasks that look automatable but require super-human capability, and questions that need investigation before anyone commits to a build, all in one budget line. That mix is how a campaign budget burns through a quarter.
The dividing line is plain. “We want the AI to do this repeatable task instead of us” has far better odds of measurable ROI than “we want the AI to be smarter than us”.
Brynjolfsson, Li and Raymond (NBER, published in the QJE in 2025) studied 5,179 customer support agents and found a generative AI assistant raised issues resolved per hour by 14% on average, by 34% for novice and low-skilled workers, with minimal impact on the most experienced staff. Their suggestive explanation is that the model spreads the practices of the best agents to newer ones. Gains are not automatic everywhere, though: another study reported Danish workers using chatbots saved 2.8% of their time on average, with no significant change in earnings or hours.
Co-pilots and agents are different investments
A co-pilot (one prompt, one response) has a single-inference cost structure, so each use carries one countable cost. An agentic workflow plans, retries and orchestrates across systems on its own, so the cost per outcome varies from run to run and payback stretches to one to three years or more. We would start most businesses on co-pilot use, where a human stays in the loop and the unit economics are legible, and fund agents only once a process is understood well enough to redesign. Where agents actually stand in production is a separate question, covered in our look at what agentic AI is actually shipping versus what is still research; industry by industry, the co-pilot use cases across industries show where this split plays out.
Where does generative AI still need a human reviewing the output?
Human review stays wherever a wrong answer costs money or creates liability. Generative models can produce inaccurate claims, citations or summaries that sound confidently correct, which is what hallucination means. McKinsey’s State of AI 2024 survey found about 44% of organisations had experienced at least one negative consequence from generative AI, with inaccuracy and hallucination among the most cited risks.
Prototypes hide this. In customer service, the notebook demo handles the case beautifully, and production then shows hallucination, latency, drift and silent failure on edge cases the prototype never saw. Retrieval-augmented generation, which pulls relevant documents in at inference time and feeds them into the response, is how many applications ground answers in a company’s own knowledge. Retrieval narrows errors rather than removing the need for a reviewer, because a confidently wrong answer built on the right document is still wrong.
The liability point is not theoretical. In Moffatt v. Air Canada (2024 BCCRT 149), Air Canada argued it could not be held liable for information given by its chatbot, a position the tribunal called “a remarkable submission”, and the passenger was awarded $650.88 in damages. The tribunal held that a company must take reasonable care that its representations are accurate and not misleading.
The EU AI Act sets oversight rules for high-risk systems. Article 14 of the EU AI Act requires high-risk AI systems to be designed so they can be effectively overseen by natural persons while in use. The system must enable its overseers, as appropriate and proportionate, to understand its limitations, monitor it, stay aware of automation bias, interpret its output, and override or safely stop it. Article 14 covers high-risk systems specifically, and the oversight measures must match the risk, autonomy and context of use.
A practical rule: keep review where output reaches a customer, a regulator or a contract, and loosen it where output is internal, reversible and checked downstream anyway.
What does it cost a business to run generative AI, and how do you measure the return?
The formula is the ordinary one: ROI = (financial return − total investment) ÷ total investment × 100. What makes generative AI hard is that both sides of that equation misbehave.
On the cost side, the investment includes inference API spend across providers, GPU compute, vector databases, data pipelines and the engineering time holding it together, most of which never appears as an “AI” line item. CloudZero’s ROI in the AI Era report found organisations budget 30% to 36% of cloud spend for AI while AI-specific line items show only 2.5%, and that reported AI spending runs roughly 12 times lower than actual AI-driven cloud consumption. The FinOps Foundation lists the drivers as compute, data storage and transfer, model training and model serving, including versioning, scaling and monitoring.
On the return side, time saved is the trap: hours saved become ROI only when they turn into shipped work, avoided headcount or revenue.
Worked example: the same project at 367% and at 110%
CloudZero’s worked example shows how hidden costs change the answer. With a return of $1,512,000 against a stated investment of $324,000, ROI is ($1,512,000 − $324,000) ÷ $324,000 = 367%, with payback in under three months. Add the spend hiding in the cloud bill (GPU capacity for the embedding pipeline, preprocessing jobs, logging and evaluation infrastructure, a shared Kubernetes cluster) and assume a fully loaded $60,000 a month, giving $720,000: ROI becomes ($1,512,000 − $720,000) ÷ $720,000 = 110%, with payback closer to six months. It is still a good project; the damage is that the board approved 367%.
Unit metrics that make the return defensible
Tokens are the unit of cost most generative AI workloads can track and attribute to individual use cases, and input tokens are typically billed at a much lower rate than generated tokens. Counting API requests instead is inaccurate, since a two-token “hello” and a 1.2-million-token document cost the same under that method, though it can serve as a quick fix for some workloads. Reserved capacity complicates things: provisioned throughput guarantees availability and low latency for significant upfront cost, but cannot scale down overnight when usage drops. Effective per-token rates quoted for reserved capacity assume 100% utilisation, which is rarely the case.
The FinOps Foundation is blunt that token consumption alone does not show value delivered: a cheap model doing many tasks may deliver nothing useful, while a costlier one may do work the cheap one cannot, so cost has to be validated against quality, accuracy and performance. The measurable unit is cost per business outcome, such as per resolved conversation or per processed document, checked against quality. Per-request serving cost and GPU sizing are engineering questions under that number, not the business case itself.
Model choice moves the cost more than people expect
The FinOps Foundation’s sentiment-analysis scenario makes this concrete. Hosting Llama 2 7B needed only 24GB VRAM instances on Hugging Face or SageMaker, about $500 for the expected usage; Llama 2 13B needed 80GB or 96GB instances at about $2,500; Llama 2 70B needed 160GB or 192GB instances at $4,700 to $7,000. In that scenario, a Llama 2 7B endpoint was among the cheapest options and GPT-4 was “incredibly expensive”, to the point where the same money could run a small load-balanced cluster of 7B or 13B models and still save several thousand dollars a month. Quantisation then cut the 13B hosting cost from about $2,500 to about $500, and quality held for that use case after retesting. The authors stress that the result belongs to their scenario and changes as its parameters change.
Payback by use case
| Use case | Average payback | Return profile | Measurement difficulty |
|---|---|---|---|
| Customer support deflection | 3 to 9 months | Cost avoidance, countable per ticket | Low: cost per resolved conversation |
| Back-office document processing | 3 to 9 months | Cost avoidance and error reduction, highest measured ROI per MIT | Low: cost per document |
| Coding assistants | 3 to 12 months | Productivity; ROI only if shipping speed or headcount changes | Medium: cost per engineer vs output |
| Marketing and content | 6 to 12 months | Volume and speed; revenue link indirect | Medium-high: attribution contested |
| Agentic workflows | 12 to 36 months | Process redesign, higher ceiling, multiplied inference cost | High: retries blur cost per outcome |
Scope changes the timeline. Single use cases with countable units commonly pay back in 3 to 9 months. Enterprise-wide programmes take longer: Deloitte’s 2025 survey found most organisations reach satisfactory ROI on a typical AI use case in 2 to 4 years, and only 6% see payback in under a year.
Why most generative AI pilots show no P&L impact
The MIT GenAI Divide report behind the 95% figure, as Fortune reported it in August 2025, is that about 5% of AI pilots achieve rapid revenue acceleration while the large majority stall with little or no measurable P&L impact. It rests on 150 interviews with leaders, a survey of 350 employees and an analysis of 300 public deployments. The Google Cloud figures measure something else: among enterprises reporting revenue gains, 86% saw growth of 6% or more. When reading executive surveys, note that Federal Reserve researchers say senior leaders may now face pressure to report AI use as an efficiency initiative.
MIT attributes the failures to a learning gap, not model quality. Generic tools like ChatGPT work well for individuals because they are flexible, but stall in enterprise use because they do not learn from or adapt to workflows. CloudZero’s summary of the same research adds pilots disconnected from real workflows and budgets aimed at the wrong functions. More than half of generative AI budgets go to sales and marketing tools, yet MIT found the biggest ROI in back-office automation: cutting business process outsourcing and external agency costs and streamlining operations.
Who builds it matters too. In Fortune’s account of the MIT study, buying from specialised vendors and building partnerships succeeded about 67% of the time, while internal builds succeeded only a third as often. CloudZero’s summary of the same study says external solutions succeeded roughly twice as often, so the two accounts agree on the direction but not on the size of the gap. Line managers driving adoption, rather than only a central AI lab, and tools that integrate deeply and adapt over time also separated the successes.
There is a structural lesson under the statistics. An engagement whose only value arrives at the end, with a perfect final output, loses momentum and usually fails; each milestone has to produce something usable even before the end goal is reached. A deflection bot that handles the simplest 20 intents well is a product; a promise to handle everything next year is not. For the technical side, see our breakdown of GenAI-specific failure patterns.
Build, buy or run it on your own hardware
The FinOps Foundation’s paper compares closed-source services, hosted open-source models and DIY cloud deployment; the on-premise row draws on the arXiv cost-benefit analysis below.
| Option | Strengths | Weaknesses | Fits when |
|---|---|---|---|
| Commercial closed API | Rapid deployment, high-quality models, support | Limited customisation, pricing risk, shared bugs and biases | Early R&D and low volume, where per-token billing is cheapest |
| Hosted open-source model | Control, privacy compliance, competitive inference pricing | Scarce expertise; quality, guardrails and observability are your job | You need control over prompts, weights or fine-tuning |
| DIY on cloud AI services | Full control of models and data, fits existing cloud estate | Expertise needed, longer development and debugging | The workload must sit inside existing security and operations tooling |
| On-premise | Data stays in-house, suits regulated domains | Hardware upfront plus running costs | High volume or strict data residency |
A 2025 cost-benefit analysis on arXiv found break-even typically within a few months for small models, about 2 years for medium models and about 5 years for large ones, making on-premise viable mainly at 50M tokens a month or more, or under strict residency rules. Small open models under 30B parameters ran on a single consumer RTX 5090 (about $2k) and could break even against commercial APIs in under three months. Medium models ran on two A100-80GB GPUs (about $30k) with accuracy typically within 10% of large ones. Against aggressively priced providers, though, break-even for large models stretched to 5 to 9 years, weak unless privacy, sovereignty or lock-in override cost. Medium-volume organisations benefit most from hybrids: sensitive work local, burst traffic to cloud APIs.
The authors add the caveat that matters most: prices and models move so quickly that deployment is a long-term investment to revisit, not a one-off choice.
How can a business avoid lock-in to one generative AI vendor, cloud or hardware platform?
Avoid lock-in by deciding, at the start, which artefacts must stay portable (your data, your model and your deployment) and building them on open formats, before switching is ever on the table. The ITEA journal’s 2025 review of AI procurement defines lock-in as proprietary architectures, data formats or integration dependencies that limit the ability to switch suppliers, cut costs or adapt systems. Its scale shows in the 2021 USDA case, where the department paid $112 million more for Microsoft Office than Google Workspace to avoid even higher switching costs.
The forms it takes, and the matching mitigations:
- Platform lock-in: training, inference and deployment tied to one ecosystem through custom APIs and vendor-specific services such as Bedrock, Azure OpenAI Service or Vertex AI. OCI-compliant containers on Kubernetes give a portable environment that runs on any compliant cluster, and Kubernetes-native serving such as KServe works across cloud providers.
- Data lock-in: custom formats that resist migration. Open table formats such as Apache Iceberg, Delta Lake and Apache Hudi separate storage from compute so workloads can move without replicating data.
- Model and framework lock-in: the review recommends designing models for export through ONNX from the start, avoiding operations that resist conversion, because the deployable artefact, not the training framework, should decide portability. We would still benchmark the exported model on the actual target rather than assume parity.
- Operational lock-in: containers, ONNX, MLflow and Infrastructure-as-Code together form the technical base for keeping at least two qualified sources.
- Contractual lock-in: the review’s model clauses give the buyer unlimited rights to its training data, derivative datasets, and models trained on its data, including weights. That language is written for government buyers, but the principle carries to any business.
- Expertise lock-in: if only one vendor’s people understand how the system is built and run, the switching cost is knowledge, so documentation and handover belong in the deliverables.
European law now helps on the cloud side. Chapter VI of the EU Data Act requires cloud and edge providers to meet minimum requirements for interoperability and switching. Platform and Software as a Service providers must offer open interfaces and export data in a commonly used, machine-readable format. Switching and data egress charges are removed entirely from 12 January 2027, with providers allowed to charge costs in the transition period from January 2024.
Portability has a cost of its own, so we scope it against a named trigger: a performance ceiling, a deployment-target constraint, or a lock-in cost the business can put a figure on. A model that performs in its training environment is a different artefact from one that performs on the deployment target, and closing that gap is engineering work rather than a hand-off.
The decision that comes before vendor and model
Run the steps in order. First, does the task sit within what current models do reliably? Then, can you count its outcome in P&L terms with the full cost attached? Is a human reviewing what reaches customers? Does each milestone leave something usable behind? Finally, could you move the workload if prices or providers change? If you cannot name the unit you are paying for and the unit of value it produces, the right next step is measurement, not a bigger model.
