Most GenAI teams spend their effort in the wrong place. They tune the model, rewrite the prompt, swap the base checkpoint, and chase a few points on a benchmark. Meanwhile the thing that actually decides whether the system survives contact with production — the data it learns from and is evaluated against — sits untouched, accepted as a fixed input rather than treated as engineering work. That inversion has a name. A data-centric approach holds the model steady and systematically improves the data: its coverage, its labeling consistency, its representativeness of what production will actually send. It is the opposite of the model-centric and prompt-centric habits most teams default to. And it is usually the difference between a prototype that demos well and a system that keeps working after launch. What does a data-centric approach mean in practice? The term gets used loosely, so it helps to be concrete. A data-centric workflow treats three things as first-class, auditable artifacts: Coverage — does your dataset actually contain the distributions, edge cases, and rare-but-critical inputs production will produce? Not “roughly,” but measurably. Label consistency — when two annotators (or two prompts, or two reference answers) disagree, is that disagreement measured and resolved, or does it silently degrade the training and evaluation signal? Representativeness — is the data you tuned and evaluated on drawn from the same population as production traffic, or from a cleaner, friendlier slice someone curated to make the demo work? In a model-centric loop, you improve the score by changing the model. In a data-centric loop, you freeze the model and improve the score by fixing the data — finding the mislabeled examples, filling the coverage gaps, and re-measuring on a held-out set that looks like production rather than like the demo. Andrew Ng’s data-centric AI framing popularized this for supervised learning; the same discipline transfers directly to generative systems, where the “labels” become reference outputs, rubrics, and evaluation prompts. None of this replaces good modeling. It reorders the effort. When a system is already near the frontier of what the base model can do, further model tuning yields diminishing returns while data work often has room left. Deciding which lever has slack is itself an engineering judgment — one we make explicit rather than assume. For the broader framing of what this discipline is and where it fits a feasibility decision, see our data-centric approach to AI and what it means in practice for feasibility. Why does a GenAI prototype that works on curated data fail on production data? Here is the mechanism, and it is not mysterious once you name it. A prototype is almost always built and evaluated on curated data — examples someone selected, cleaned, and often unconsciously chose because they were tractable. The model learns those distributions and is scored against them. On that data it can look finished. The divergence point is production, where three things the curated set quietly guaranteed stop being guaranteed: Distribution shift. Production traffic contains inputs the curated set under-represented — different phrasings, formats, languages, domains, and lengths. The model was never asked to handle them, so its behavior there is untested rather than good. Edge cases at real frequency. The rare-but-critical inputs that a curated set trims for cleanliness are exactly the ones that generate the loud failures — the hallucinated citation, the leaked PII, the confidently wrong answer on the one query that mattered. Label and rubric drift. For generative outputs, “correct” is defined by reference answers and evaluation rubrics. If those were built inconsistently on curated data, the prototype’s accuracy number is measuring against a moving target that doesn’t hold in production. The observable symptom is a gap: prototype accuracy on curated data versus observed accuracy on held-out, production-representative data. In our experience across GenAI engagements, teams who never measure that gap discover it in production, at the most expensive possible stage — after launch, after the budget is spent, after users have seen the failure (an observed pattern from delivery work, not a benchmarked rate). A data-centric workflow shrinks the gap before launch by building the held-out set to look like production and by fixing coverage and labels until the two numbers converge. The spoke on the data-centric approach in practice for fixing GenAI data-quality failures walks through the remediation loop in detail. Data-centric vs model-centric vs prompt-centric: which lever moves what The three postures are not competitors so much as answers to where the slack is. The table below is a decision aid, not a ranking — the right lens depends on which resource your system is actually short of. Dimension Model-centric Prompt-centric Data-centric What you change Base model, fine-tuning, architecture Instructions, few-shot examples, decoding Coverage, labels, representativeness of train/eval data What stays fixed Data, prompt Model, data Model, prompt Best when Base model genuinely lacks capability Model capable, framing suboptimal Prototype–production gap is large; failures are input-specific Failure it masks — Under-tested distributions look “solved” by clever prompting — Cost profile GPU-heavy, slow iteration Cheap, fast, but ceilings quickly Labor-heavy up front; compounding return What it cannot fix Bad data → confidently learns the wrong thing A genuine capability gap A base model that simply cannot do the task The trap is that prompt-centric work is cheap and fast, so teams over-invest there. Clever prompting can make an under-tested distribution look solved on the demo set while doing nothing for production coverage. That is precisely how a prototype earns confidence it hasn’t earned. When you find yourself adding your fifth few-shot example to handle a failure class, the honest read is usually that the data — not the prompt — is the constraint. What does a data readiness audit check before a prototype is built? The corrected approach is to run a data readiness audit before prototyping, not cleanup after a demo. It converts an implicit assumption — “someone accepted this data” — into an explicit, auditable decision. Use this as a diagnostic checklist: Provenance & rights. Where did every dataset come from, and are you permitted to train and deploy on it? This gates everything downstream. Coverage map. Enumerate the input categories production will produce and check each against the dataset. Empty cells are your known failure modes, listed before launch instead of discovered after. Label/rubric consistency. Measure inter-annotator (or inter-rubric) agreement. Low agreement means your accuracy number has a wide, unmeasured error bar. Our note on data labelling and annotation as the data-quality gate behind GenAI failure covers how to run this measurement. Production-representative held-out set. Build an evaluation set drawn from real (or realistically simulated) production traffic, kept separate from anything the model touches. This is the instrument that measures the gap. Drift plan. Production data moves. Decide, before launch, how you will detect distribution shift and re-audit — which connects directly to how to monitor ML models in production. Named owner of acceptance. Someone always accepts the data, explicitly or by default. Name that person. Data-centric discipline is largely the practice of making that acceptance a decision rather than an accident. This audit is the data-readiness component of a GenAI feasibility assessment — the check that catches the data-quality-blindness failure pattern before it consumes budget. Running it early is the whole ROI argument: it converts a class of failures normally caught at expensive post-launch stages into decisions made before spend. How do you measure data quality when the outputs are open-ended? This is the hard part of generative use cases and where the discipline earns its keep. With a classifier you have a label and an accuracy number. With open-ended generation there is no single right answer, so measurement has to be constructed deliberately. Three practices tend to hold up. First, rubric-based scoring: define what “good” means as explicit, testable criteria — factual grounding, format compliance, absence of harmful content — and score outputs against the rubric rather than against a golden string. Second, reference-set evaluation with LLM-as-judge, calibrated against human ratings so you know the judge’s own error rate; frameworks like this are increasingly standard, but a judge you haven’t calibrated is just another unmeasured assumption. Third, coverage-weighted evaluation: report accuracy per input category, not as a single blended number, so a strong average can’t hide a category that fails completely. A blended 92% that is 99% on common inputs and 40% on a critical edge case is not a 92% system in any way that matters. For teams standing this up, tooling matters less than discipline, but it helps to version the evaluation data the same way you version code — data version control tools for production GenAI exist precisely because an evaluation set that silently changes invalidates every comparison you draw from it. When is a data-centric approach the wrong lens? Restraint matters here, because “just fix the data” can become its own dogma. A data-centric approach does not help when the base model genuinely cannot do the task — no amount of curation teaches a model a capability it structurally lacks, and pouring labeling effort into an impossible task is a slower, more expensive way to fail. It also does not resolve problems that are really product or specification problems: if nobody has defined what a good output is, no dataset can encode it. And it does not substitute for latency, cost, and infrastructure feasibility — a model that answers perfectly but too slowly or too expensively is still infeasible, which is why data readiness is one component of a feasibility assessment, not the whole of it. The general failure-mode framing — that most AI projects fail on data and problem definition long before modeling — is broader than GenAI. If you want that wider lens on why AI projects fail and how to de-risk them at the R&D stage, that is a separate methodology thread; this article keeps the focus strictly on the GenAI-specific mechanism. FAQ What’s worth understanding about a data-centric approach first? A data-centric approach holds the model steady and systematically improves the data it learns from and is evaluated against — its coverage, label consistency, and representativeness of production. In practice you freeze the model, find and fix mislabeled or missing examples, fill coverage gaps, and re-measure on a held-out set that looks like real traffic rather than the demo set. It reorders effort rather than replacing modeling: you spend where the slack actually is. How does a data-centric approach differ from the model-centric or prompt-centric effort most GenAI teams default to? Model-centric work changes the model (fine-tuning, architecture) while data stays fixed; prompt-centric work changes instructions and examples while the model and data stay fixed; data-centric work changes the data while the model and prompt stay fixed. The trap is that prompt-centric iteration is cheap and fast, so teams over-invest there — clever prompting can make an under-tested distribution look solved on the demo while doing nothing for production coverage. The right lens depends on which resource the system is actually short of. Why does a GenAI prototype that works on curated data fail on production data, and how does data-centric work close that gap? Prototypes are built and scored on curated data that someone selected and cleaned, so the model is only tested on distributions the curated set contained. Production introduces distribution shift, edge cases at real frequency, and label or rubric drift the curated set quietly hid, so the observed accuracy drops. A data-centric workflow builds a production-representative held-out set and fixes coverage and labels until prototype and production numbers converge — before launch, instead of discovering the gap after spend. What does a data readiness audit actually check before a GenAI prototype is built? It checks data provenance and usage rights, a coverage map against the input categories production will produce, label and rubric consistency (measured, not assumed), a production-representative held-out evaluation set, a drift-detection plan, and a named owner accountable for accepting the data. Each empty coverage cell is a known failure mode listed before launch rather than discovered after. It is the data-readiness component of a GenAI feasibility assessment. How do you measure data quality and coverage for a generative use case where outputs are open-ended? Because there is no single right answer, you construct the measurement: rubric-based scoring against explicit, testable criteria; reference-set evaluation with an LLM-as-judge calibrated against human ratings so you know the judge’s own error rate; and coverage-weighted evaluation that reports accuracy per input category rather than as one blended number. A strong average can otherwise hide a critical category that fails completely. At what point in a GenAI project should data-centric work happen, and who is accountable for accepting the data? Data-centric work belongs before and during development — a data readiness audit before prototyping, then continuous re-measurement, not cleanup after a demo. Running it early converts failures normally caught at expensive post-launch stages into decisions made before spend. Someone always accepts the data, explicitly or by default; data-centric discipline names that owner and makes the acceptance an auditable decision. When is a data-centric approach the wrong lens, and what data problems does it not solve for GenAI? It does not help when the base model genuinely lacks the capability — no curation teaches a model something it structurally cannot do. It does not resolve product or specification gaps where nobody has defined what a good output is, and it does not substitute for latency, cost, and infrastructure feasibility. That is why data readiness is one component of a feasibility assessment rather than the whole of it. If your GenAI prototype demoed well and then degraded after launch, the question to ask is not which model to try next — it is whether anyone ever measured the gap between your curated evaluation set and your production traffic. That gap is the data-quality-blindness failure pattern, and a data readiness audit inside a GenAI feasibility assessment is where you catch it before the budget is gone.
