We treat a generative AI use case as technically feasible when five checks pass on your own data and in your own workflow: the data passes a data readiness check, the task genuinely needs generation, the accuracy ceiling sits above what the task needs, the remaining errors are tolerable or reliably caught, and latency and cost per request fit the workflow. Check them in that order, because a failure early in the list makes the later checks pointless. Public benchmarks cannot settle any of this. They answer “what can the model do?”, while a deployment decision needs to know whether this workflow is fit, reliable, safe and worth scaling on your data and under your controls. The evidence comes from a pilot that tests a frozen configuration against thresholds set before results are seen.
There is a question that comes before the checklist: is the outcome you expect plausible at current capability levels? “We want an AI to be smarter than us” is a very different proposition from “we want AI to do this repeatable, menial task instead of us”, and the second is far more likely to produce a return you can measure. TechnoLynx works on problems that create value at realistic intelligence levels and does not commit to outcomes that would need super-human capability. The checks below are how that judgement gets tested in practice.
The feasibility check, in order
| Order | Check | Question it answers | Stop or rethink when |
|---|---|---|---|
| 1 | Data readiness | Is there complete, representative, current data, and can someone write a gold-standard answer for the task? | No expert can author a correct reference output, or the data misses the cases the system will meet |
| 2 | Technique fit | Does the task need generation at all, or would rules or a fine-tuned classifier do it? | A heuristic or a fine-tuned small model already meets the requirement |
| 3 | Accuracy ceiling | Can the best available configuration reach the accuracy the task needs? | The task cannot carry the 5–10% factual inconsistency rate a practitioner reports even after grounding |
| 4 | Error tolerance | What does one mistake cost, and will review actually catch it at volume? | Errors are costly and the review design has not been measured |
| 5 | Latency and cost | Does response time and cost per request fit the workflow? | Response time or cost per response exceeds what the workflow accepts |
| 6 | Pilot gates | Did a frozen configuration clear thresholds set in advance? | Results land below the minimum viable threshold |
The order is our judgement, not a standard. Data comes first because every later measurement is taken on that data, and a pilot run on unrepresentative data produces numbers that do not describe production.
Data readiness comes first, and there is no universal pass mark
Data readiness for AI means preparing and checking the quality, accessibility and suitability of datasets before an AI application uses them. The survey literature lists the dimensions to assess: quality, completeness, representativeness, feature relevance, duplication, class separability, FAIR compliance and outliers. A wider catalogue for structured data adds mislabels, class imbalance, correctness, timeliness, privacy leakage and sample size.
Four of them deserve a closer look here. Completeness is whether the required data and attribute values are actually present. Duplicates over-represent particular records and distort the patterns a model learns or retrieves. Models trained on imbalanced data favour the majority class and perform poorly on the minority class, which is often the one you care about. Timeliness is whether the data still reflects current conditions. For a retrieval-backed assistant answering from a policy library, timeliness tends to be the dimension that decides it: a correct answer drawn from a superseded document is still a wrong answer.
No universal threshold marks data as “ready”. The authors of the 2024 data readiness survey say universal, context-independent threshold values are hard to define, given how context-dependent data is and how varied AI use cases are. They also note that specific applications often need customised readiness metrics and domain expertise. So the useful question is narrower: is this data ready for this task at this level of risk?
Evaluating a generative use case adds one more requirement. Someone has to be able to write the correct output for each task. The EnterpriseVal framework (2026) assumes a gold artefact can be authored, and notes that this holds for generation and classification workflows but less so for open-ended advisory tasks. If your experts cannot agree on what a good answer looks like, the pilot has nothing to grade against, and that counts as a feasibility finding in its own right.
Where the system falls under the EU AI Act’s high-risk category, data readiness is also a legal duty. Article 10 requires training, validation and testing data to be relevant, sufficiently representative and, as far as possible, free of errors and complete for the intended purpose. It also requires governance practices that cover data collection and origin, preparation steps such as labelling and cleaning, an assessment of whether the needed data is available, in sufficient quantity and suitable, an examination for bias, and the identification of data gaps and how they will be addressed. Our view is that the same list works as a readiness template outside the high-risk scope too. Organisational readiness (skills, ownership, budget) is a separate question; see our guide to assessing enterprise AI readiness before a project starts.
When is generative AI the wrong tool, and rules or classic ML the better fit?
Generative AI is the wrong choice when a rule, a heuristic or a trained classifier already does the job. Google’s People + AI Guidebook says that a rule- or heuristic-based solution often works as well as an AI version, or better, and is easier to build, explain, debug and maintain.
Classification is one documented case. Bucher and Martini (2024) compared ChatGPT (GPT-3.5 and GPT-4) and Claude Opus, prompted zero-shot, against smaller fine-tuned models on sentiment, emotion, approval and party-position tasks across news, tweets and speeches. Fine-tuning on application-specific training data gave better performance in every case. If the task is to route, tag or sort, and labelled data exists, a fine-tuned small model is the baseline a generative approach has to beat.
Some task shapes are documented weak spots. A 2023 survey of LLM evaluation reports limited abstract reasoning and poor performance on natural language inference. The same survey reports that LLMs can fabricate facts and struggle to include real-time or changing information. The 2025 complexity study reports a study in which top-tier LLMs lost more than 60% of their performance on elementary reasoning problems when a subtle condition was changed, which fits the view that they recite familiar solution templates. When a use case depends on precise logic over changing inputs, deterministic code with a model on the language edges is usually the safer design.
How high is the accuracy ceiling for this task?
The accuracy ceiling is the practical limit on how precise a generative system can be, and it tightens as the system’s scope and generality grow. OpenAI’s 2025 technical report, as summarised by the authors of a 2025 complexity study, recorded the o3 model hallucinating on 33% of questions in its internal PersonQA benchmark, about double the 16% recorded for o1. A study reported in the evaluation survey found GPT-4 and BingChat answered more than 80% of factual questions correctly, still more than 15% short of full accuracy.
Grounding helps, but it does not remove the problem. Practitioner Eugene Yan puts the typical factual inconsistency or irrelevance rate at 5 to 10% even after RAG grounding and careful prompt engineering, and reports that LLM providers find it may be prohibitively hard to get below 2%. That range is a useful sanity check for a feasibility discussion: a task that needs well under 2% unsupported output is unlikely to be feasible without guardrails on the output. Lower figures have been measured on well-scaffolded work. EnterpriseVal reports 1.6 to 3.2% hallucination on real wholesale-credit documents under human grading, with retrieval and citation scaffolding. Neither figure transfers to your corpus, and that is why the pilot exists.
How much error can the use case tolerate, and will human review catch the rest?
Error tolerance depends on what a single mistake costs, not on the average error rate. Dhar’s Decision-Automation Map, discussed in the 2025 complexity study, plots predictability against cost per mistake. Low-consequence tasks such as spam filtering have long been automated, while high-cost errors make full automation impractical even when residual inaccuracy is modest. Yan makes the same point for evaluation bars: internal uses such as product classification or document summarisation carry lower risk than a customer-facing medical or financial chatbot.
The tempting fix is to put a human in the loop and assume the reviewer absorbs the errors. In a randomised trial of AI-trained physicians (medRxiv preprint, 2025), the group given flawed LLM recommendations in half the cases reached a mean diagnostic accuracy of 73.3%, an adjusted 14.0 percentage points below the control group. The authors attribute this to automation bias, the tendency to accept automated output without enough scrutiny. Microsoft Research’s 2022 literature review of around 60 papers describes overreliance as users accepting incorrect AI outputs, which matters because policymakers and practitioners call for oversight that makes users the last line of defence against AI failures.
Review also has a cost that can wipe out the benefit. The value of an assistant depends on verification taking much less effort than creating the work from scratch, and the authors of a 2025 legal education study found that carefully reviewing the AI’s subtly flawed answers took them longer than writing correct answers themselves. EnterpriseVal treats this as a design rule: human-in-the-loop is a component whose detection performance is measured, not an assumption that turns an imperfect system into a safe one. Our position is that a feasibility study should measure the reviewer’s catch rate and minutes per item at expected volume, and treat a review design that has not been measured as an unknown.
Latency and cost per request as feasibility constraints
Latency and operating cost limit where a model can be deployed at scale, and both grow with model complexity. The 2025 complexity study cites a two-second extra delay for a typical GPT-4 chat response compared with GPT-3.5, and reports that the conversational platform Rasa moved to smaller models after higher latency increased user dissatisfaction. On cost, Andreessen Horowitz analysts estimate that generating a million tokens with a frontier model costs roughly 20 to 40 times as much as with a mid-tier alternative under current cloud pricing.
The same study suggests a simple grid for each workflow: how quickly must the system respond, and what is an acceptable cost per response? Often the answer is a hybrid, where lightweight models handle routine queries and larger models are kept for high-stakes or ambiguous cases. A feasibility case fails here when the accuracy needed calls for the large model, while the volume and margin only allow the small one. Building that number properly needs a load-aware model, which we cover in building a cost-per-request model for inference at scale.
Technical feasibility is not economic viability
A use case can pass every technical check and still not be worth building. A 2026 assessment framework separates the two explicitly: operation, quality assurance and integration create costs that can reduce the achievable benefit, and economic viability weighs development, operation, control and external dependencies against expected value. The same authors note that the quality of a single output is not enough for business use. Economic relevance increases when a service can be delivered repeatedly, integrated into existing systems and scaled at acceptable cost.
This is why we favour repeatable, menial work. Google PAIR’s guidance points the same way: automate tasks that are difficult, unpleasant or need scale, ideally where the people doing them agree on the correct way to do them. The 2025 complexity study observes that measurable productivity gains appear first where accuracy errors matter least. A task that is repetitive, has an agreed right answer and has a low cost per error lines up all of these conditions at once. A vague target such as “make our analysts smarter” is far less likely to produce measurable return, a theme of why generative AI projects fail before they launch.
How should a feasibility pilot be scoped?
Scope the pilot around one workflow, a frozen system configuration and a task sample drawn from real work. Avoid open exploration of what the model can do. EnterpriseVal defines the use case as tasks sampled from the workflow population, each with permitted sources and an expert-authored gold artefact, plus a rubric, an autonomy level and a consequence tier. The configuration under test is frozen. It includes the model, prompts, retrieval, tools, guardrails and the human review roles, and any change to it triggers re-evaluation. Including the review design matters, because two deployments of the same model with different review designs are different systems.
A workable sequence, adapted from the EnterpriseVal lifecycle:
- Charter the use case: autonomy level, consequence tier and the value hypothesis.
- Set the thresholds for each gated metric before any results are unblinded, with the business owner, model owner and risk function in the room.
- Draw a stratified task sample. At the lightest evaluation intensity, the framework asks for at least 100 graded units.
- Author gold artefacts and a rubric, with key elements a competent answer must contain.
- Run the frozen configuration with repeats and logs.
- Grade blind, with at least two practitioner graders per task and a third to adjudicate.
- Apply the gate and the value model.
Setting thresholds first also has a practical benefit. The EnterpriseVal authors report that writing thresholds first exposed an inverted pair, a self-grading step and an unstated baseline. A pilot is also only a pre-deployment snapshot. The 2024 auditing blueprint notes that pre-deployment techniques cannot fully anticipate risks in systems that change over time, which is why post-deployment monitoring still has to follow. A polished demo is not a pilot, a distinction we develop in what to evaluate in a development partner beyond demo quality.
What pilot evidence justifies moving toward production hardening?
The evidence that justifies production hardening is a frozen configuration clearing pre-agreed thresholds on gated metrics, with reported sample sizes and confidence intervals, graded by people whose agreement was checked. EnterpriseVal gives each gated metric two thresholds. The outer one is the minimum viable level, tied to the harm of failure and the existing controls. The inner one is the level at which the use case may scale without extra controls. Results between the two fall in a conditional zone: deploy with enhanced review, sampling or restricted scope, and a dated commitment to reach the target. The gap between thresholds should be at least the expected sampling uncertainty at the planned sample size.
The bank pilot reported in that paper gives an example of results clearing their gates. In credit-memo drafting, human-graded citation precision reached 88% and the hallucination rate 1.6% for the best model, against gates of 70% and 5%. A metric only entered the gate decision when grader agreement reached a Cohen’s kappa of 0.6. Below that, the rubric was revised, not the system. Gate only the metrics tied to a material harm, because a conjunctive gate rejects more often as the number of gated metrics grows.
Three things weaken a “go” decision, in our judgement: thresholds chosen after the results, a sample too small to separate the minimum from the target, and a review step that was assumed rather than measured. When the evidence passes, the next work is production hardening: post-deployment monitoring of outputs and procedures for handling failure modes, which the auditing blueprint lists as part of application assurance.
FAQ
How do you tell if an AI system is generative?
Generative AI is a functional definition: the system outputs content such as text, images, audio or code. A system that only assigns labels or scores, such as a spam classifier, is not generative even if it uses a large model internally.