Separating shipped from speculation in agentic AI Agentic AI has attracted an unusual volume of announcement-driven coverage that conflates early research, private betas, and production deployments. For teams evaluating whether and how to adopt agentic systems, the gap between “demonstrated in a controlled setting” and “running reliably in production” is the critical distinction — and the gap is wider than the press releases suggest. This article tracks what is actually deployed, what sits in constrained pilots, and what remains primarily research as of mid-2026. The framing matters because procurement and architecture decisions made on marketing categories rather than engineering realities produce systems that look impressive in demos and fail in production. For the underlying conceptual distinction between an agentic system and a generative one, see our companion piece on agentic AI vs generative AI. What is shipping in production? The table below is an observed-pattern summary from how we see agent-class systems deployed across enterprise contexts. It is not a benchmarked ranking — it is a snapshot of where the engineering risk has been driven down enough for production use. Category Status Examples Caveat Code-generation assistants Widely deployed GitHub Copilot, Cursor, Codeium Bounded scope: suggestions within editor context Customer-service automation Deployed at scale Airline / telco tier-1 support Narrow domains, high human fallback rates RAG-based knowledge workers Deployed in enterprises Document Q&A, internal search Quality depends heavily on retrieval quality Code review and test generation Deployed in CI pipelines PR summarization, test scaffolding Reliability varies by language and framework Workflow automation with tool use Constrained deployment CRM data entry, scheduling Requires constrained action spaces The common thread in what is actually working: constrained scope, bounded action spaces, and reliable human fallback paths. Agents that can take any action in an open-ended environment are not in reliable production use at scale — observed pattern across the engagements we have seen, not a benchmarked rate. What is in controlled pilots Several categories appear frequently in announcements but remain in constrained enterprise pilots: Multi-agent research pipelines. Systems where specialized agents conduct literature review, generate hypotheses, and draft sections. Working in some pharmaceutical and academic contexts with heavy human review. Not autonomous. Software-development agents. Agents that file issues, write code, submit PRs. Working in limited scope — bug fixes in well-tested codebases. Failure rate remains high for open-ended feature work, which is why most of these systems still require a developer-in-the-loop. We cover the scope question in more detail in our note on autonomous AI software engineering capabilities. Autonomous browsing and data extraction. Agents that navigate web interfaces to collect data or complete forms. Technically feasible but brittle against UI changes — a redesign of a target site routinely breaks pipelines that worked the previous week. What remains primarily research Long-horizon planning with reliable goal decomposition beyond roughly five steps. Self-improving agents that modify their own reasoning processes. Multi-agent systems coordinating effectively on complex open-ended tasks without human intervention. Reliable tool-use chaining across diverse, untested APIs. These items are not exotic — they appear in many announcements — but the published academic benchmarks and the small number of public production case studies do not yet show reliable behaviour in any of them. Treating them as available capability when scoping a project is the most common failure we see in early agentic AI procurement. How do you evaluate agentic AI claims? When assessing an agentic AI product or announcement, the following questions cut through most of the noise: What is the action space? Narrow (fill form, send email) versus open-ended (do anything on the web) determines reliability more than model choice. What is the human-fallback rate? Systems that advertise 95% automation often run materially lower in production conditions — observed pattern across enterprise pilots, not a published-survey figure. What happens on failure? Does the agent halt and escalate, or does it take incorrect actions silently? What is the evaluation benchmark? Demos on cherry-picked tasks, internal benchmarks, and published academic benchmarks have very different reliability implications. What does production look like? A deployment at one enterprise with heavy configuration is not evidence of general deployability. The most reliable current deployments share a pattern: they work within a small, well-defined action space, have clear failure modes, and route to humans for anything outside that scope. That pattern is mundane compared to the announcements, which is precisely why it works. What distinguishes production-ready agent systems from demos? The gap between an impressive agent demo and a production-ready agent system is primarily about failure handling, cost control, and evaluation methodology — not the core AI capability. Production-ready agents need explicit failure modes. A demo can retry indefinitely or fail gracefully with an error message. A production agent handling customer requests must distinguish between retriable failures (API timeout, rate limit), non-retriable failures (impossible request, missing permissions), and partial successes (completed three of five requested actions). Each failure type requires a different response: retry with backoff, inform the user with an explanation, or report partial completion with options for the remaining actions. Cost control separates production from prototype. A demo agent can call an LLM API dozens of times to reason through a complex request. A production agent processing thousands of requests per day must bound its cost per request. We implement token budgets per request (maximum input plus output tokens across all LLM calls), step budgets (maximum number of tool calls per request), and latency budgets (maximum wall-clock time before the system returns a response, even if incomplete). These caps are unglamorous but they are what makes the difference between a viable unit economics and a runaway bill. Evaluation methodology is the third gap. Demos are evaluated on cherry-picked examples. Production agents need systematic evaluation: accuracy on a representative test set, latency distribution across request types, cost per request by complexity tier, and failure rate by failure category. In our experience, evaluation datasets of 200–500 requests covering the full range of expected use cases, re-run after every code change, catch the regressions that human testing routinely misses because the test set exercises edge cases that testers rarely think to try. The model capability is usually not the bottleneck. GPT-4-class models — and the open-weights families that have caught up since — are capable enough for most agent tasks today. The engineering around the model (tool integration, error handling, cost management, systematic evaluation) determines whether the system is production-ready. Teams that misallocate their effort towards model choice rather than this surrounding scaffolding tend to produce demos that never make the jump. For the conceptual frame underneath all of this — what makes a system “agentic” rather than generative in the first place — our piece on LLM agents explained walks through the orchestration layer in more detail. FAQ What is agentic AI, and how is it engineering-distinct from generative AI? Agentic AI is an orchestration layer: a system that plans actions, calls tools, observes results, and iterates toward a goal. Generative AI produces outputs (text, code, images) from a single inference call. The engineering distinction is that agentic systems require state management, tool integration, failure handling, and cost control around the model — generative systems mostly require good prompting and reliable serving. Is ChatGPT a generative AI or an agentic AI — and why does the distinction matter for scoping? Plain ChatGPT is generative. Modes that browse, call tools, or execute code are agentic. The distinction matters because the second category needs explicit budgets, failure modes, and evaluation harnesses that the first does not. Scoping a project as “generative” when it is in fact agentic is a common cause of cost and reliability surprises. What are concrete examples of agentic AI versus generative AI in real workflows? Generative examples: drafting a marketing email, summarizing a document, generating a code snippet. Agentic examples: an assistant that reads a customer ticket, looks up the order in the CRM, issues a refund through an API, and emails the customer. The agentic example involves multiple tool calls, state, and failure paths; the generative example does not. How does the infrastructure for an agentic system differ from a generative one? Agentic systems need tool registries, action-space definitions, per-request budgets (tokens, steps, latency), structured logging of each step, retry and escalation logic, and an evaluation harness that exercises full workflows rather than single completions. Generative systems can be served with a prompt template and a model endpoint. When does a use case need an agent, and when is a single generative call sufficient? If the work can be done in one inference — produce this text given that input — a single generative call is sufficient and far cheaper to operate. If the work requires looking things up, taking actions, or iterating based on results, you need an agent. The cost difference between the two is large enough that defaulting to agentic when generative would do is a common waste. How do agentic AI, generative AI, and predictive AI fit into one architecture without overlapping? Predictive AI scores or classifies. Generative AI produces content. Agentic AI orchestrates — often calling predictive and generative models as tools. A clean architecture treats predictive and generative components as services with stable contracts, and the agent layer as the thing that decides which service to call and what to do with the result.
