An operator asks the assistant why the line-side inspection model flagged the last batch. If the answer comes from an unversioned document pile, it is a guess dressed up as a citation. The retrieval corpus, not the model, decides whether that answer is honest — and that is the part most teams get wrong when they wire a retrieval-augmented generation (RAG) layer onto an industrial computer-vision stack. The appeal is obvious. You have thousands of pages of runbooks, drift reports, calibration notes, and post-incident writeups scattered across wikis and shared drives. A large language model with retrieval promises to make all of it queryable in plain language. The naive build is to index every PDF, embed everything, and trust the model to surface what matters. That build works in a demo and fails in production, because the failure it produces is not “no answer” — it is a confident answer sourced from stale context. When the line lighting or fixturing changes, an ungrounded assistant keeps citing the retired configuration as if it were current. What does RAG architecture for an LLM actually do here? RAG is straightforward in mechanism. A user query is embedded into a vector, the system retrieves the nearest chunks from an indexed corpus, and those chunks are stitched into the model’s prompt as context before it generates an answer. The model does not “know” your inspection stack; it paraphrases whatever the retriever hands it. That is the whole design, and it is also the whole risk. The quality ceiling of a RAG assistant is set by the governance of its corpus, not by the size of its model. For a line-side inspection assistant, the corpus is not a knowledge base in the marketing sense. It is a set of reliability artefacts that already exist to keep an industrial CV model in service: rollback runbooks, drift telemetry summaries, sensitivity-calibration evidence, and model-version pinning records. These are the same artefacts that let an on-call owner recover from a drift incident. The RAG layer does not replace them — it retrieves them. If those artefacts are versioned and dated, the assistant degrades honestly. If they are dumped in flat, the assistant hallucinates a green light. We see this divergence over and over: two teams build what looks like the same RAG stack, one grounds it in governed artefacts and the other in an ungoverned drive, and only the first one survives contact with a real fixturing change. Why a flat document dump fails, and versioned artefacts do not The core problem is provenance. When you embed every document without version metadata, retrieval optimises for semantic similarity, not currency. A query about “why the model rejects this connector orientation” will retrieve the most textually relevant chunk — which might be a calibration note written for a model version retired four deployments ago. The retriever has no way to prefer the current artefact, because nothing in the index tells it which artefact is current. Versioned, dated artefacts fix this at the retrieval layer, not the generation layer. Each chunk carries the model version it describes, the date it was written, and a status flag. Retrieval can then filter to the version actually in service before ranking by similarity. The model still paraphrases, but it paraphrases the right source. This is why the corpus has to be treated as a governed artefact subject to the same discipline as the rest of the pipeline — the reliability gates that decide what enters production should also decide what enters the retrieval index. Our view on where reliability gates belong across an ML pipeline applies directly: the index is a pipeline stage, and it needs a gate. Two contrasts make the difference concrete. Dimension Flat document dump Versioned artefact corpus Retrieval key Semantic similarity only Version filter, then similarity Answer when config changes Cites retired configuration confidently Filters out retired context, degrades honestly Provenance in the answer None or fabricated Version pin + date attached to each cited chunk Incident-context reconstruction Manual cross-checking of every source Drift history retrieved with its pins Failure mode Confident hallucination Missing-context flag or “no current source” The right-hand column is not a bigger model. It is a corpus wired to the version-pinning evidence that already keeps the inspection model trustworthy. This is where the industrial-CV validation lens belongs: the RAG corpus is only trustworthy when its sources are the validated, version-pinned reliability artefacts, not an ungoverned pile. How do you keep answers tied to the model version in service? The mechanism is version-aware retrieval, and it has three moving parts. First, every artefact ingested into the index carries a version pin — the exact model version it describes — as structured metadata, not free text buried in a paragraph. Second, the retriever queries a live source of truth for “which version is in service right now” before it ranks anything, and applies that as a hard filter. Third, the generation prompt is constructed so the model must cite the version pin of each source it draws on, which means a mismatch surfaces in the answer rather than hiding inside it. The load-bearing idea is that the version pin lives in the metadata and drives the filter. A common shortcut is to put the version in the document text and hope the model notices — it will not reliably notice, because embedding similarity does not privilege a version string. Store it as a queryable field. Tools like a vector store fronted by a metadata filter (pgvector with SQL predicates, or a managed store with attribute filtering) make this a first-class operation rather than a hopeful post-processing step. The same discipline that governs line-side CV telemetry and incident logs in a full-text search layer applies to the RAG index: the store has to answer version-scoped queries, not just similarity-scoped ones. What retrieval-provenance checks belong in the validation pack? If the corpus is a governed artefact, it needs validation of its own. The checks below are the retrieval-side equivalent of the coverage and calibration evidence you already produce for the inspection model. Treat this as a diagnostic rubric before a RAG-grounded assistant goes near an operator. Version-filter enforcement. Confirm the retriever rejects out-of-version chunks under a fixturing-change scenario, not just under a happy-path query. Test the failure case explicitly. Provenance attachment. Every generated answer must carry the version pin and date of each cited source. An answer without provenance is a validation failure, not a stylistic nit. Stale-source detection. Verify that when no current-version source exists for a query, the assistant returns a missing-context flag rather than reaching for the nearest retired artefact. Ingestion gating. Check that only validated, version-pinned artefacts enter the index — the index build is a pipeline stage subject to the same gate as a model promotion. Retrieval-quality scoring. Measure whether the retrieved chunks are the ones a human on-call owner would have pulled for that incident. This is a benchmark-class check when you score it against a fixed query set with known-good answers. Anchoring these checks to the validation lens is the point: the retrieval-quality and provenance scoring sit under the same scorecard discipline as the rest of the reliability pack. A RAG architecture built specifically for perception-validation evidence retrieval runs the same provenance checks against a different corpus — the mechanism transfers even when the artefacts differ. How a grounded RAG layer shortens drift recovery Here is the ROI, stated plainly and bounded to what it actually touches. When a drift incident fires, the on-call owner’s first job is reconstructing context: which model version is live, what changed recently, what the drift telemetry shows, and what the rollback runbook says. Done manually across scattered sources, this is an hours-long scavenger hunt. A RAG layer grounded in versioned artefacts retrieves drift history and rollback steps with their version pins already attached, which is an observed pattern across the reliability engagements we run — it compresses that reconstruction from hours to minutes rather than being a benchmarked figure. It also reduces mis-attributed root-cause calls, because the assistant’s answers stay tied to the version in service instead of a retired one. What it does not do — and this boundary matters — is change rejection-rate accuracy. That stays owned entirely by the inspection model. The RAG layer is a recovery-time and context-fidelity tool, not an accuracy tool. Conflating the two is exactly the overclaim that erodes trust in operator-facing assistants. The metric the parent reliability pack is measured on is recovery time from drift incidents, and that is the metric a grounded RAG layer moves. FAQ What should you know about rag architecture llm in practice? A RAG-augmented LLM embeds the user’s query, retrieves the nearest matching chunks from an indexed corpus, and inserts them into the model’s prompt as context before generating an answer. The model paraphrases the retrieved sources rather than recalling facts from training. In practice this means the answer’s quality is capped by the governance of the corpus, not the size of the model — for a line-side assistant, the corpus must be the versioned reliability artefacts, not a flat document dump. Why must a RAG corpus for a line-side inspection assistant use versioned, dated artefacts instead of a flat document dump? A flat dump ranks retrieval by semantic similarity only, so a query can surface a calibration note written for a retired model version because nothing in the index marks which artefact is current. Versioned, dated artefacts let the retriever filter to the model version actually in service before ranking by similarity, so the assistant degrades honestly instead of citing a retired configuration confidently. How do you keep RAG answers tied to the inspection model version currently in service? Store each artefact’s version pin as structured, queryable metadata rather than free text; query a live source of truth for the in-service version and apply it as a hard filter before ranking; and construct the generation prompt so the model must cite each source’s version pin, surfacing any mismatch in the answer. Putting the version only in document text is unreliable because embedding similarity does not privilege a version string. What retrieval-provenance checks belong in the validation pack for a RAG-grounded assistant? Version-filter enforcement under a fixturing-change scenario, provenance attachment on every answer, stale-source detection that returns a missing-context flag when no current source exists, ingestion gating so only validated version-pinned artefacts enter the index, and retrieval-quality scoring against a fixed query set. These are the retrieval-side equivalent of the coverage and calibration evidence produced for the inspection model. How does a grounded RAG layer shorten recovery from a drift incident without touching rejection-rate accuracy? It retrieves drift history and rollback steps with their version pins already attached, compressing the on-call owner’s context reconstruction from hours to minutes and reducing mis-attributed root-cause calls. It does not alter rejection-rate accuracy — that stays owned by the inspection model — so the reliability metric it moves is recovery time from drift incidents, not detection accuracy. What happens to a RAG assistant’s answers when the line lighting or fixturing changes, and how do you prevent stale context? An ungrounded assistant keeps citing the retired configuration confidently because retrieval has no signal that the old artefact is stale. A version-aware corpus filters out chunks pinned to the retired version, so the assistant either draws on the current artefact or returns a missing-context flag — preventing stale context by making currency a retrieval filter rather than a hope. The open question for most teams is not whether to add RAG — it is whether their reliability artefacts are governed well enough to ground it. If the runbooks, drift summaries, and version pins are not already versioned and dated, the assistant will inherit that gap and amplify it. Production hardening defines the versioned artefacts a RAG corpus must retrieve to stay honest about the model actually in service; the retrieval layer is only as trustworthy as the artefacts feeding it.
