The failure rate is high, but not random Gartner predicted in 2018 that through 2022, 85% of AI projects would deliver erroneous outcomes due to bias in data, algorithms, or the teams responsible for managing them — a directional industry-scale estimate (market-direction) that subsequent industry analyses have broadly tracked. McKinsey’s published surveys report that only around a fifth of companies deploying AI at scale see significant financial impact from those investments (published-survey). VentureBeat’s commonly-cited figure that roughly 87% of data science projects never reach production is another published-survey claim, with looser methodology. The specific percentages vary by definition, but the directional finding is consistent: most enterprise AI projects do not deliver their intended business outcome. The failure rate is not random. The failures cluster around a small number of predictable patterns — patterns that are identifiable before the project begins, during the scoping phase when the investment commitment is signed. The organisations that succeed at enterprise AI do not have better models or better data scientists. They have better project selection, clearer success criteria, and more realistic scoping. Generative AI projects face these same patterns alongside their own specific failure modes; GenAI projects frequently fail before they launch due to scope inflation, evaluation gaps, and demo-to-production underestimation. What follows is a working taxonomy of root causes — the ones we see repeatedly in our consulting engagements, not the ones that show up in conference talks. Why does data readiness cause the most failures? The most common root cause of enterprise AI project failure — and the most underestimated during scoping — is data readiness. A model requires data. The data must exist, be accessible, be clean, be representative, and be available in sufficient volume. Each of these requirements fails independently and frequently. The data does not exist. A demand forecasting project requires 24 months of point-of-sale data by SKU and location. The organisation has aggregate monthly sales by category. The gap is not bridgeable by model sophistication. We have walked into engagements where this single observation cancelled a six-figure project in week one. The data exists but is not accessible. It lives in a legacy system without an API, in a third-party platform with licensing restrictions, or in departmental silos where data sharing needs governance approvals that take months — sometimes longer than the planned project duration. The data exists and is accessible but is not clean. Missing values, inconsistent formatting, duplicate records, and stale entries degrade model performance in ways that are not obvious until training. Across our data-readiness engagements we routinely see projects where the majority of engineering effort goes into data cleaning rather than modelling (observed pattern, not a benchmarked industry rate) — and the project was scoped assuming the data was ready. The data is not representative. Training data reflects historical patterns that no longer hold. A fraud detection model trained on 2019 transactions performs poorly on 2024 patterns because customer behaviour, merchant types, and fraud methods have shifted. The fix is a data readiness assessment before the project is committed — not a desk-research data audit that lists datasets, but a hands-on evaluation that examines the actual data quality, coverage, and accessibility against the requirements of the proposed model. In practice this means pulling a real sample through a notebook, profiling it with pandas or Spark, and checking whether the columns the modelling team is counting on are populated, consistent, and joinable. Pattern 2: Success criteria are not defined “We want to use AI to improve customer service.” What does “improve” mean? Reduce average response time? Increase first-contact resolution rate? Reduce staffing cost? Increase customer satisfaction scores? Each of these is a different project with different data requirements, different model approaches, and different integration needs. Projects without specific, measurable success criteria cannot be evaluated — and projects that cannot be evaluated cannot be course-corrected. The team builds something, the stakeholders look at it, and the judgement is subjective: “this doesn’t seem right” or “I expected something different.” Without predefined criteria, the project enters an indefinite iteration cycle with no convergence criterion. The discipline is to define success criteria before development begins: specific metrics (reduce average response time from four hours to one hour), measurement methodology (from ticket creation to first response, or from first response to resolution?), and acceptance thresholds (the model must hit this metric at this level for the project to be considered successful). The phrasing matters; we have seen the same metric defined three different ways across business, data, and engineering teams in the same kickoff meeting. Pattern 3: Integration is underestimated An AI model produces a prediction. For that prediction to have business impact, it must be delivered to the right person, at the right time, in the right system, with the right context. This is integration — and it is consistently the most underestimated component of enterprise AI projects. The model that detects fraud has to be integrated with transaction processing to block suspicious transactions in real time. The model that predicts equipment failure has to push into the maintenance scheduling system. The model that classifies customer inquiries has to route tickets through the existing ticketing platform. Each integration involves API development, data format translation, error handling, authentication, latency management, and testing against the production system — usually wrapped in a container orchestrated by Kubernetes, fronted by a serving framework like Triton or a custom FastAPI service, and instrumented for monitoring through whatever stack the SRE team already runs. In our experience, integration work accounts for the majority of total project effort on serious enterprise deployments (observed pattern, not a benchmarked industry rate). Plans that budget 80% for model development and 20% for integration are systematically inverted. The GenAI prototype-to-production gap is a specific instance of this general pattern: the prototype demonstrates model capability, but production engineering — integration, monitoring, guardrails, cost management — is the bulk of the remaining work. Pattern 4: The problem does not require AI Not every business problem that involves data requires a machine learning model. A rule-based system, a well-designed dashboard, a process improvement, or a simple statistical analysis may solve the problem more reliably, more cheaply, and more quickly. A project to “predict which customers will churn” may discover, on inspection, that the top three churn predictors are: the customer called support more than five times in the last month, the customer’s contract is in its final 30 days, and the customer received a price increase. Those rules can be implemented in a CRM workflow in a day. As an illustrative example from our consulting engagements (observed pattern, not a benchmarked rate), the ML model that predicts the same outcome with around 78% accuracy took three months to build and now requires ongoing maintenance. The evaluation question is whether the business problem genuinely requires the adaptive, data-driven decision-making that AI provides — or whether a simpler approach delivers the same outcome. An AI solution is appropriate when the decision is complex (too many variables for explicit rules), when the patterns are non-obvious (relationships that humans cannot detect by inspection), or when the scale of decisions exceeds what human review can sustain (millions of transactions, documents, or customer interactions). Where do the headline 85–95% failure rates actually come from? The numbers that travel furthest in AI failure discourse — 85% from Gartner’s 2018 prediction, 87% from VentureBeat, MIT-cited figures in the 90s — come from different methodologies measuring different things. Gartner’s figure was a forecast about erroneous outcomes through 2022, not a measured failure rate. VentureBeat aggregated industry commentary about projects “never reaching production,” which conflates cancelled projects with shelved POCs and with deployed-but-ignored models. MIT Sloan Management Review’s surveys ask executives about perceived business value, which is a different construct again. A serious team should not internalise any single number. The defensible internal heuristic is: assume the base rate of an unscoped AI project delivering measurable business value is well below 50%, and that the rate inside your organisation depends on your project-selection discipline, not on the industry average. The four patterns above are the actionable substitute for the headline number. Failure-pattern decision rubric Use this rubric as a pre-commitment checklist. Each row is a red flag — any one of them is sufficient to pause the investment for a structured assessment. Pattern Diagnostic question Red flag Data readiness Has someone actually opened the data and profiled it? “We’re confident the data is there” with no notebook output Success criteria Is success defined as a number with a measurement method? Success described in qualitative terms (“better”, “smarter”) Integration Is each downstream system named with an effort estimate? Integration is a single line item in the plan AI necessity Has a non-AI baseline been costed and benchmarked? Project initiated because “we need to use AI” Sponsorship Is there an accountable business owner, not just an IT sponsor? Project owned by the data team with no business counterpart POC-to-prod path Is there a defined pivot point with criteria to kill or scale? POC scoped without a stated success-to-scale criterion This rubric is what an AI Project Risk Assessment formalises. Every failed project we have reviewed exhibited at least one of these red flags at inception — before any code was written. Organisational versus technical failure A useful split: organisational failures (scope, sponsorship, data ownership, success definition) versus technical failures (model selection, infrastructure, monitoring). In our engagement experience, organisational failures dominate. Technical problems are usually solvable once the project has a defensible scope, a defined success metric, and a real data pipeline. Organisational problems are not solvable by the engineering team — they require someone with authority to redefine the project or stop it. This is why enterprise AI projects survive POC but die between POC and production. The POC is a technical exercise that the data team can complete on its own. Production deployment requires organisational commitments — integration owners, SLA agreements, on-call rotations, cost allocation — that surface the absence of organisational sponsorship that was masked during the POC. For generative AI projects specifically, evaluating use case feasibility before building applies the same logic to GenAI-specific challenges — hallucination tolerance, RAG quality requirements, and cost-at-scale projections. Scenario: the predictable £400K failure A mid-sized logistics company committed £400,000 to an AI-driven route optimisation system. The project triggered every failure predictor above — and none were assessed before the budget was approved. No one had examined the actual GPS and delivery data; when the team finally did, 40% of historical route records had missing waypoints and inconsistent timestamps (operational measurement from that project, benchmark class). Success was defined as “better routes” with no target metric for delivery time reduction or fuel savings. Integration with the fleet management system was a single line item — £30K — that ultimately required £140K of API development, real-time data pipeline work using Kafka and a custom ETL layer, and driver-app modification. A rules-based alternative using three scheduling heuristics, tested in a two-week pilot, later achieved roughly 80% of the projected benefit at under £25K (operational measurement from that pilot, benchmark class). The original project was cancelled at month six after consuming £380K. A structured assessment against the four failure predictors — conducted in two weeks for under £15K — would have identified each of these issues and redirected the investment toward the rules-based approach. What a sober AI project looks like The organisations that watch their peers fail and then quietly succeed share a few traits. They commission a data readiness assessment before authorising the model build. They define success as a specific delta against a measured baseline, not as “deploying AI.” They scope integration work as a first-class engineering programme, not an afterthought. They keep a non-AI baseline running in parallel so they can decide on evidence rather than enthusiasm. And they treat the AI project as a business-risk decision with an accountable owner, not as an IT initiative delegated to whoever is closest to the data. None of this is glamorous. It is, however, the consistent shape of the projects that survive contact with production. If projects in the pipeline have not been evaluated against these failure patterns, an AI Project Risk Assessment identifies which are likely to succeed and which should be restructured or cancelled before the investment accumulates. FAQ Why do most enterprise AI projects fail, and which root causes are not the ones publicly discussed? Most enterprise AI projects fail at four predictable patterns: data readiness gaps that scoping missed, success criteria defined qualitatively rather than as measurable deltas, integration treated as a line item rather than the majority of the work, and applying AI to a problem a rule-based system would solve more cheaply. The publicly discussed failures — model accuracy, talent scarcity, infrastructure cost — are usually downstream effects of these earlier scoping choices. Organisational misalignment, where the project is owned by IT without a business counterpart, is the less-discussed root cause that quietly determines most outcomes. Where do MIT and Gartner’s reported failure rates (85–95%) actually come from, and what is the right number for a serious team to internalise? Gartner’s widely-cited 85% figure was a 2018 forecast about erroneous outcomes through 2022, not a measured project-failure rate. VentureBeat’s 87% number aggregates industry commentary about projects never reaching production. MIT Sloan surveys executive perception of business value, a different construct again. The defensible internal heuristic is to assume the base rate of an unscoped AI project delivering measurable value is well below 50%, and that the rate inside a specific organisation depends on its project-selection discipline rather than on any industry headline. Which failures are organisational versus technical? Organisational failures — scope ambiguity, missing sponsorship, undefined success criteria, unclear data ownership — dominate in our engagement experience. Technical failures around model choice, infrastructure, and monitoring are usually solvable once the project has defensible scope and a real data pipeline. Organisational failures are not solvable by the engineering team; they require someone with authority to redefine or stop the project. Why do enterprise AI projects survive POC but die between POC and production? A POC is a contained technical exercise the data team can finish on its own. Production deployment requires organisational commitments — integration owners, SLA agreements, on-call rotations, cost allocation — that the POC did not need. When those commitments do not exist, the project stalls in the gap between demonstration and deployment. The absence is structural, not technical. What does a sober AI project look like in an organisation that has watched its peers fail? It commissions a data readiness assessment before authorising the model build, defines success as a specific delta against a measured baseline, scopes integration as a first-class engineering programme, runs a non-AI baseline in parallel for honest comparison, and assigns an accountable business owner. The project is treated as a business-risk decision with explicit pivot points, not as an IT initiative. How is general enterprise AI failure different from the GenAI-specific failure patterns covered elsewhere? The four general patterns — data readiness, success criteria, integration, AI necessity — apply to all enterprise AI projects, traditional ML included. GenAI projects inherit those and add their own: hallucination tolerance, retrieval quality requirements, evaluation harnesses that work for open-ended outputs, and cost-at-scale projections that traditional ML projects do not face. The general patterns are necessary conditions; the GenAI-specific patterns are additional ones that apply on top.
