The AI evaluation metrics that matter for a generative system before release are the ones that check each generated claim against the source it should come from: claim-level groundedness, and the hallucination rate that follows from it. Fluency and overlap scores such as ROUGE and BERTScore measure textual similarity, not truth, so a fluent answer with an invented number can still score well. If an LLM judge does the checking at scale, it has to be validated against human labels first. The pass/fail threshold is a risk decision, set per workflow and backed by enough annotated samples to mean something.

The essential hallucination evaluation metrics and what each may decide

Metric	What it measures	Role in a release decision
Claim-level groundedness (faithfulness)	Supported claims divided by total claims, checked against the source or retrieved context	Primary gate
Hallucination rate	Share of responses containing at least one hallucinated claim	Primary gate, read together with severity
Judge-human agreement	Cohen’s κ and the confusion matrix between the automated checker and human labels	Precondition before a judge may gate anything
Negative-test behaviour	Whether the system says the evidence is insufficient when the answer is absent from the corpus	Hallucination-resistance test for RAG systems
Retrieval metrics	Recall@K, Mean Reciprocal Rank and NDCG on the retrieved passages	Diagnostic
ROUGE, BLEU, BERTScore	Lexical or semantic similarity to a reference	Secondary signal only

The right-hand column shows how much weight we would let each metric carry in a gate.

How do LLM evaluation metrics differ from accuracy, precision, recall and F1?

Classic accuracy, precision, recall and F1 assume each prediction can be compared with one correct label. Generated text breaks that assumption, because one answer often mixes supported and unsupported information, so a single right-or-wrong judgement on the whole answer is inadequate. One fix is to break the output into atomic facts and compute the percentage supported by a reliable knowledge source, which is what FActScore does. Conventional evals built on n-grams, semantic similarity or a single gold reference have become less effective at telling good responses from bad as tasks have become more open-ended.

Precision and recall don’t go away. They can be used to score the checker itself, as the recall and precision of an LLM evaluator at flagging factually inconsistent responses. Report both together. A detector can reach high recall simply by flagging aggressively, and Realm Labs, a monitoring vendor, points out that recall published without a false-positive rate shows only half the picture.

Why don’t ROUGE and BERTScore catch hallucinated facts?

They score overlap with a reference, and a sentence can overlap heavily with the truth while changing the one word that makes it false. The GO FIGURE meta-evaluation gives an example: swapping “decreased” for “increased” in a summary of Scotland’s renewable energy output reverses its meaning, and ROUGE-L does not change.

A human study published at EMNLP 2025 compared ROUGE and an LLM-as-judge against human consensus on 200 short-answer QA pairs selected where the two disagreed. ROUGE matched the judge’s recall of 0.957, but its precision was 0.401 against the judge’s 0.736, and its agreement with humans was 0.142 against 0.723. ROUGE flagged many correct answers as errors and systematically penalised correct answers that were simply verbose. It also gave relatively high scores to wrong answers that shared surface structure with the reference. When several established hallucination detection methods were re-scored with an LLM-as-Judge metric aligned with human judgements, their performance dropped by up to 45.9%.

Two other studies point the same way. A 2026 audit of 900 human-labelled question-answer pairs found ROUGE-L prone to false negatives and strict NLI almost entirely dominated by false positives, and oracle-tuned thresholds still left these baselines short of criterion-aligned judges. Maynez et al. (ACL 2020) measured Spearman correlations with human faithfulness judgements of 0.197 for ROUGE-1 and 0.190 for BERTScore, against 0.431 for a textual-entailment score.

A check that correlates better with faithfulness is entailment at the claim level: split the output into claims, then judge each as supported, contradicted or missing relative to the source. Entailment-based scoring of this kind can flag fabricated relationships or dropped qualifiers that overlap metrics miss. Entailment has its own weakness. It needs no reference, so it can be gamed: the first sentence of any source document is always entailed by the document. For that reason Maynez et al. recommend coupling it with a reference-based measure such as ROUGE.

How do you measure hallucination rate against ground-truth data?

The procedure below is the core of the measurement. Versioning the test set and rerunning it on every candidate model are harness-design questions; see designing task-specific evaluation harnesses for LLM deployment decisions.

1. Fix the definition before anyone labels. Under a factual-correctness criterion, an answer counts as hallucinated if it contradicts the reference or contains a specific verifiable detail that is false. A true elaboration that is merely absent from the reference is not penalised. For summarisation and RAG the target is usually support by the source rather than truth in the world, and that choice has to be written down.
2. Build the test set from real inputs, with negative cases included. Negative cases ask for information that is not in the corpus, which tests whether the system respects the limits of its evidence. In the RAG-Tester study, real-world PDFs exposed 64.4% more failures than automatically generated ones.
3. Break each output into claims and verify each claim against the source. The RAGAS faithfulness metric uses an LLM to extract every claim in an answer and reports the percentage that can be inferred from the given context.
4. Aggregate to a hallucination rate. One common definition is the percentage of responses with at least one hallucinated claim. Vectara’s leaderboard reports hallucination rate as 100 minus the factual consistency rate on its summarisation set.
5. Record severity next to the rate. SelfCheckGPT’s annotation scheme separates major-inaccurate, minor-inaccurate and accurate sentences. A purely binary label hides partial hallucinations and differences in how severe the errors are.
6. Report the standard error of the mean alongside the rate.

Which metrics show a RAG maintenance assistant’s answers are grounded in what it retrieved?

Groundedness, scored claim by claim, shows whether a retrieval-augmented generation (RAG) answer comes from evidence: a response is grounded when its factual claims can be traced to the retrieved evidence. For a maintenance assistant, that evidence is the telemetry excerpt and the manual passage retrieved for the alert. Each recommendation should be checked against those passages, not against a generic reference answer.

TechnoLynx builds this check into a Production AI Monitoring Harness. It checks each recommendation against the retrieved telemetry and manual passages and reports the hallucination rate on safety- and cost-relevant claims.

Groundedness is not the only number a RAG pipeline needs. Retrieval quality is measured separately, with metrics such as Recall@K, Mean Reciprocal Rank and NDCG, while the generated answer is scored for faithfulness to the retrieved context. One vendor of evaluation tooling argues for keeping groundedness and faithfulness as separate scores. Groundedness means no claims beyond the retrieved documents; faithfulness means no distortion of what those documents say. A single accuracy number cannot show whether a regression came from retrieval drift or from generation drift. A recommendation that cites a vibration limit the manual does not contain fails groundedness. One that cites the right passage but inverts its limit fails faithfulness.

Two failure patterns in maintenance-style deployments need their own tests. The first is missing evidence. Here the desired behaviour is to say the evidence is insufficient rather than to produce a plausible but unsupported answer. A study of RAG over AI policy documents found that stronger retrieval sometimes led to more confident hallucinations when the relevant documents were missing: the retriever returned related but outdated material, and the generator treated it as evidence. Better retrieval metrics did not reliably translate into better end-to-end answers.

The second pattern is the multi-step agent. The Trajel audit covered 225 expert-annotated agent trajectories on industrial AssetOpsBench tasks. In one trajectory, an anomaly-detection tool returned nothing and a summarisation agent still reported a specific anomaly, a failure that evaluation of the final answer alone cannot see. The same study’s LLM judge reached an F1 of 0.855 on the yes/no question of whether a trajectory hallucinated. Yet its per-type agreement with humans fell to κ = 0.176 for referential hallucinations and 0.211 for logical ones, and the authors conclude that human review remains essential for those types. However good an automated groundedness scorer’s headline number looks, it needs periodic human audit on the subtle categories.

RAG reduces hallucination but does not remove it: models can still fabricate or misattribute information when retrieval returns incomplete or partly contradictory context. The RAG-Tester authors note that it is convenient for the judge model to differ from the model under test.

What hallucination rate is acceptable before the assistant goes near a ticketing workflow?

For unsupported claims that would trigger a safety action or a cost, the acceptable rate is close to zero. Lower-stakes outputs, such as a digest of recent alerts, can tolerate more, so the threshold should be set per recommendation category. If the measured rate on safety- or cost-relevant claims is above the agreed threshold, the integration stays blocked and the work goes back to retrieval and prompting. If the rate is below the threshold, and an evaluation independent of the team that built the pipeline confirms it, a staged rollout with groundedness monitoring in production is reasonable. Independent evaluation adds time and cost, so it is best scoped to the highest-risk categories. It should be repeated whenever the corpus, the manuals or the mix of alerts changes, because an offline rate describes the system you tested, not the one that drifts afterwards.

What catches hallucinated clinical facts in a candidate summarisation model?

A task-specific harness that checks each statement in a summary against the source clinical note catches hallucinated clinical facts. ROUGE and BERTScore measure lexical similarity rather than medical correctness, so they cannot. In healthcare evaluation, the distinctive method is claim-level factuality verification: outputs are split into atomic statements, and each is judged as supported, contradicted or missing relative to the source data.

TechnoLynx’s LLM Selection Pack uses this kind of harness to compare candidate models. The first step is agreeing with your clinical reviewers what counts as a hallucinated clinical fact, and which review workflow sits around the metric.

A validated instrument exists to anchor the rubric. The PDSQI-9, an LLM-centric adaptation of the PDQI-9, scores nine attributes. It was designed so that its Accurate attribute captures hallucinations and its Thorough attribute captures omissions. Fluent clinical summaries can contain hallucinated facts or leave out critical information, which is why fluency cannot stand in for either attribute. ROUGE or BERTScore can still run alongside as an initial overlap signal. They should never be the gate, because overlap can stay high while hallucination is present. A harness also catches only the fact types it was built to check (medication, dose, laterality, dates, whatever the reviewers listed), so new error types need a standing route back into the rubric.

Should we use an LLM-as-judge, and how do we validate it against clinicians?

Use one when clinician review of every output is infeasible, and only after its verdicts have been compared with clinician labels on a representative sample. A 2026 scoping review of LLM judges in healthcare found Cohen’s κ against experts ranging from 0.59 to 0.88 across the ten studies that reported it, with reliability varying by task, rubric and model. The same review reports that judges have been unreliable at detecting factual hallucinations in clinical document summarisation. When both generator and evaluator are LLMs, their errors can reinforce each other rather than cancel out.

The criterion written into the judge prompt matters as much as the choice of judge model. In the 2026 audit of 900 human-labelled question-answer pairs, replacing a faithfulness-style prompt with a factual-correctness prompt improved agreement with human labels and reduced the dominance of false positives for nearly all judges. In clinical summarisation the target is usually support by the note itself, so the prompt has to state that rule explicitly, including how to treat content that is true but absent from the note. As a benchmark for good agreement, two human annotators in that audit reached κ = 0.808 on a 300-item overlap set. One practitioner guide aims for judge-to-human agreement to match human-to-human agreement. Recall on defects is the figure to watch. One study summarised by Eugene Yan found an LLM evaluator identified over 95% of consistent summaries but only 30 to 60% of inconsistent ones.

The agreement study should report κ together with the full confusion matrix, because summary metrics alone cannot tell a judge that misses true positives from one that over-flags. Measurement protocol choices, such as how abstentions are handled and how verdicts are pooled across criteria, moved one judge’s reported accuracy from 0.551 to 0.899 without changing a single verdict. That is why the protocol should be fixed before anyone looks at the judge’s outputs. For the calibration sample, Realm Labs suggests 200 to 500 human-labelled traces, with the threshold chosen to meet a precision target of around 0.7. Re-validation is needed whenever clinical standards change, and periodically as models and workflows evolve.

What hallucination rate should disqualify a candidate?

No universal rate should be borrowed. The disqualifying threshold is a clinical-risk decision, taken in a safety review and tied to the downstream review workflow. A regulatory workshop report on AI as a medical device recommended defined thresholds for acceptable hallucination levels, set before regulatory approval and monitored after the product reaches market. They also recommended metrics that capture downstream impact and clinical relevance as well as a percentage. Severity has to be part of the threshold: a low rate of high-harm hallucinations can be worse than a higher rate of trivial ones. The same report also shows the trade-off with omissions. In a clinical question-answering test, a baseline GPT model produced 6 major hallucinations in 80 tests (7.5%), while a system with strong hallucination guardrails produced none but omitted information in 35 of its 333 tests (10.5%). Participants argued that missing information can be as harmful as a hallucination, especially when clinicians do not know it is missing.

How do you set a pass/fail threshold on a hallucination metric that you can defend?

A defensible threshold has three parts. The number comes from the risk of the workflow. The sample is large enough that the confidence interval on the measured rate sits clearly on one side of that number. And there is a monitoring rule for after release. Release gates on hallucination and grounding metrics are a reliability artefact, distinct from model accuracy, of the kind that catches regressions before customers do.

Published numbers are starting points to adjust, not standards.

Source	Suggested starting point	What kind of number it is
Openlayer (2026), vendor	Flag RAG responses below roughly 85% groundedness for review	Vendor guidance
Openlayer (2026), vendor	Alert when hallucination rate exceeds 2-3% in regulated customer-facing systems, 5-8% for internal tools	Vendor guidance
Openlayer (2026), vendor	Flag below 0.80 groundedness, block below 0.70 faithfulness, adjusted for your corpus	Practitioner starting thresholds
Realm Labs (2026), vendor	Alert above 2% hallucination rate in regulated domains, 5% for internal tools; groundedness target above 0.80	Vendor guidance
Microsoft Foundry	Default pass threshold of 3 on a 1-5 groundedness scale	Tool default

None of these numbers was derived from your failure costs, and a tool’s default threshold is a configuration value, not a risk policy.

To make the pass/fail call statistically justified, report the rate with its standard error. When comparing candidates, run inference on question-level paired differences rather than on the two headline rates, as Miller (2024) recommends. A practical rule is to pass a candidate only when the upper end of its 95% confidence interval sits below the threshold, which makes sample size part of the policy. ARES uses prediction-powered inference to combine a small human-labelled set with judge predictions on a much larger set, producing 95% confidence bounds on answer faithfulness.

Expect the offline number to be optimistic. Realm Labs estimates that pre-deployment evaluation can understate production hallucination rates by 2x to 5x depending on the domain, because real users phrase and chain requests in ways no test set anticipates. That range is a vendor’s estimate rather than a measured result for your system. The same vendor recommends alerting on change against a rolling baseline, such as a 50% rise over a 7-day baseline sustained for 24 hours, rather than relying on a static cutoff alone. How much validation evidence a given deployment needs, from a light check to a full package, is a risk-tiering question covered in validating models before they go live, from risk tiering to evidence packages.

How many annotated examples before comparing two models’ hallucination rates means anything?

Plan on a few hundred labelled items at minimum, and size the set from the smallest difference between models you need to detect. The ARES authors found that with fewer than about 100 to 150 human-labelled datapoints their method could not meaningfully tell RAG systems apart. They suggest roughly 150 to 300, and note that specialised domains such as medicine need annotators with domain expertise. Card et al. (EMNLP 2020) showed that the most common human-evaluation design at EMNLP 2019 (3 workers, 100 items) was underpowered unless the effect was large. They also found that significant results from underpowered tests tend to exaggerate the true effect or even reverse its sign. Their advice is to run a power analysis before evaluating, and to collect more data if the expected improvement will not exceed the minimum detectable effect.

Two corrections are easy to miss. When test items come in related groups (several questions per manual, say, or per patient), the standard errors have to be clustered, and Miller (2024) found the adjustment reached up to 3X in a real example. A validated judge can also reduce the human labelling budget. One 2026 analysis showed that with a judge-human R² of 0.70 and 2,000 judge-scored items, only 65 human reviews were needed to reach an effective sample of 200. If the pilot overstates R², the design allocates too few human reviews and its precision guarantees no longer hold. In safety-relevant domains, expert ratings remain the standard the judge is measured against.

Worked example: what 300 labels can and cannot separate

Assume two candidate models, each scored on 300 independent items, with true hallucination rates of 5% and 3%. These are assumed values, used only to show the arithmetic. The standard error of a proportion is √(p(1−p)/n), so the 5% rate carries about ±2.5 percentage points at 95% confidence (√(0.05 × 0.95 / 300) ≈ 0.0126, multiplied by 1.96). The standard error of the difference is √(0.0475/300 + 0.0291/300) ≈ 0.016, so a two-point gap is about 1.25 standard errors and would not reach significance. Scoring both models on the same 300 items and analysing paired differences narrows that interval, because item difficulty shared by both models cancels out.

Who can build a hallucination evaluation harness and release gate for our LLM application?

TechnoLynx builds task-specific hallucination evaluation for generative systems: claim-level groundedness checks against your own sources, an LLM judge validated against your reviewers’ labels, and the threshold your release gate runs on. When choosing a partner, ask to see the agreement study and confusion matrix behind any judge they propose, and ask whether the threshold is tied to your own risk review. For choosing between candidate models, the engagement is the LLM Selection Pack. For gates and monitoring around a live system, it is the Production AI Monitoring Harness. Both sit alongside TechnoLynx’s generative and agentic AI R&D work.

TechnoLynx runs this evaluation work through LynxBenchAI, starting from your own sources and your reviewers’ labels.

The threshold policy itself stays with your clinical safety or engineering risk owner, whoever does the measurement.

A low hallucination rate from an unvalidated checker

The figure that should worry a release reviewer most is a low hallucination rate produced by a checker nobody validated. Overconfident judge scores can create false reassurance and hide fluent but incorrect outputs. In safety-critical settings, false negatives let unsafe outputs through without human review. The harder question is usually not which metric to compute. It is who owns the threshold, and whether the agreement study behind the checker would survive the same scrutiny as the model it is grading.

FAQ

What does a hallucination detector do?

A hallucination detector flags an individual output as unsupported, incorrect or fabricated, often before it reaches a user. Hallucination monitoring, by contrast, tracks failures across many outputs over time.

Can you stop an LLM from hallucinating completely?

No approach prevents it completely: RAG and checking outputs against retrieved sources reduce hallucination but do not eliminate it. Because the research treats hallucination as theoretically inevitable, the emphasis falls on detection, mitigation and continuous human oversight.