A supplier submits a material declaration, an IMDS export, and a PPAP packet. Someone on your team has to turn that pile into review-ready compliance evidence an OEM auditor will accept. It is tempting to hand the whole pile to a model and ask for finished output — and that instinct is where most document-intelligence projects quietly acquire the risk they will pay for during an audit. Document intelligence is not a single model that reads inputs and emits compliance evidence. It is a pipeline of distinct stages — ingestion, extraction, classification, and reconciliation — each of which either preserves or destroys the link between an extracted fact and the supplier document it came from. The difference between a defensible compliance posture and a latent finding is not model quality. It is whether traceability survives each stage. What does working with document intelligence involve in practice? In practice, document intelligence means turning unstructured or semi-structured supplier submissions into structured, verifiable facts that a downstream process can act on. The phrase gets used loosely to cover everything from a scanner with optical character recognition to a large language model asked to “summarize this PPAP.” Those are not the same activity, and treating them as interchangeable is the first mistake. The useful mental model is a chain of transformations. A raw supplier document enters at one end. A set of asserted facts — each tagged with what it is, where it came from, and how confident the system is — exits the other. Every stage in between is a place where the fact either stays anchored to its origin or floats free. When a fact floats free, you have generated content that reads like evidence but cannot be defended when an OEM reviewer asks “show me where this number came from.” We see this pattern regularly: the demo works beautifully on a clean sample, and the failure only appears months later when an auditor pulls a thread and the system cannot reconstruct the provenance of a single reconciled field. What are the distinct stages of a document-intelligence pipeline? Four stages carry the load. They are worth naming separately because the automation decision — where to insert a model, where to keep a deterministic rule, where to keep a human — is different at each one. Ingestion takes the submission as it actually arrives: a scanned PDF, a spreadsheet export, an XML file from IMDS, an email attachment with the wrong extension. Ingestion normalizes format and, critically, records the source. If ingestion loses the mapping between a page region and the byte offset it came from, no later stage can rebuild it. Extraction pulls candidate facts out of the ingested content — a part number, a substance concentration, a signatory name, a date. This is where character-level and layout-aware models earn their place, because supplier documents are full of tables, stamps, and handwritten annotations that break naive text extraction. How the tokenizer treats a document affects what gets extracted; we cover that mechanism in what character tokenization means for document-intelligence extraction. Classification decides what each extracted fact is and which document class it belongs to — is this a material declaration, a control plan, a certificate of conformance? Classification errors are quiet: a field extracted correctly but filed under the wrong document type produces a reconciliation that looks complete and is wrong. Reconciliation is where extracted facts meet the compliance requirement. Does the declared substance list satisfy the OEM’s prohibited-substance rule? Does the PPAP element set match what the part requires? Reconciliation is the stage most teams want to fully automate, and it is the stage where an opaque generation step does the most damage. Which stages preserve traceability, and which risk destroying it? Traceability is a property of the link, not of any single stage. A pipeline preserves traceability when every asserted fact at the output carries an unbroken reference back to the source region of the source document. It destroys traceability when a stage substitutes a plausible value for a sourced one — the defining behavior of a generation model asked to fill gaps. The table below is the core of the decision. It maps each stage to the automation posture that keeps the source-of-truth intact. Stage-by-stage traceability decision table Stage Traceability risk Safe automation posture What to log Ingestion Low — mechanical, if source coordinates are recorded Fully automate; keep the format-normalization deterministic Source file hash, page/region coordinates, ingestion timestamp Extraction Medium — models may hallucinate a value for a low-confidence region Automate with per-field confidence; route low confidence to review Extracted value, source region, model + version, confidence score Classification Medium — a mis-typed document silently poisons reconciliation Automate, but validate class assignment against expected document set Assigned class, alternatives considered, confidence Reconciliation High — where “drafting assistance” becomes “adjudication” Assist, do not adjudicate; a human owns the pass/fail verdict Rule applied, facts consumed, verdict, reviewer identity The load-bearing rule: automation is safe wherever it reduces effort without hiding the source-of-truth. Ingestion and extraction can be heavily automated because their outputs point back at something. Reconciliation cannot be a black box, because its output is the compliance claim. This is also the reason a pipeline needs continuous monitoring rather than one-time validation. Extraction confidence and classification behavior drift as suppliers change their document formats, and traceability completeness is a metric you have to watch, not assume. Our approach to machine-learning monitoring for provenance-preserving compliance treats that drift as a first-class signal. How does document intelligence differ from OCR or a general-purpose model? Two comparisons clear up most of the confusion. Optical character recognition converts pixels to characters. It has no concept of what a field means, whether a value satisfies a rule, or which document class it belongs to. OCR is a component that can feed the extraction stage; it is not document intelligence. Treating a good OCR result as finished evidence skips classification and reconciliation entirely. We walk through where that line sits in OCR vs AI for supplier compliance documents. A general-purpose generation model sits at the opposite failure pole. It will happily produce fluent, structured output that looks like a completed compliance record — including values it inferred rather than extracted. That fluency is the hazard. A large language model asked to “produce the compliance summary” collapses extraction, classification, and reconciliation into one opaque step, and the source link dies inside it. The specific provenance risks this introduces are documented in the LLM threats to provenance in supplier compliance automation. Document intelligence, done correctly, is neither of these. It is the staged pipeline that uses models where they add lift — layout-aware extraction, document classification — while keeping the source anchor and the adjudication verdict outside any generative step. Where does drafting assistance become compliance adjudication? This is the boundary that decides whether a project is safe to ship. Drafting assistance means the system proposes and a human disposes: the pipeline surfaces extracted facts, flags likely reconciliations, and a qualified reviewer accepts or rejects. Compliance adjudication means the system itself issues the pass/fail verdict that goes into the record. The moment a reconciliation output is treated as authoritative without a human owning the verdict, you have crossed from assistance into adjudication — usually without deciding to. That crossing is rarely a design choice; it is a default that creeps in when reconciliation is automated for throughput and nobody re-examines who owns the result. In our experience, the teams that stay on the right side of this line do it by making the reviewer’s sign-off a required, logged event rather than an optional rubber stamp. The verdict has an owner, and the owner is a person, not a model. How do you evaluate accuracy, and what has to be logged? Accuracy is not one number. A document-intelligence pipeline has to be measured per field type and per document class, because a system that extracts part numbers at 99% and substance concentrations at 82% has a very different risk profile from the reverse — and a single blended accuracy figure hides exactly the fields that matter most for compliance. Useful evaluation practice (an observed pattern across regulated-document engagements, not a published benchmark): Report extraction accuracy broken out by field type, not aggregated. Report classification accuracy per document class, with the confusion between classes made explicit. Track traceability completeness — the fraction of output facts with an intact source link — as a hard gate, not a soft metric. Track reconciliation throughput and manual-review time as the ROI signals, since those are what the automation is actually buying you. What must be logged follows directly from the traceability table above. For the pipeline to remain auditable by an OEM reviewer, every output fact needs: its source document hash and region, the model and version that extracted it, the confidence score, the classification assigned, the reconciliation rule applied, and the identity of the human who owned the verdict. If any of those is missing, the audit trail has a gap the reviewer will find. FAQ What should you know about document intelligence in practice? Document intelligence turns unstructured or semi-structured supplier submissions into structured, verifiable facts. In practice it is a chain of transformations — a raw document enters, and a set of asserted facts exits, each tagged with what it is, where it came from, and how confident the system is. It is not a single model that reads inputs and emits finished evidence. What are the distinct stages of a document-intelligence pipeline? Four stages carry the load: ingestion (normalize format and record the source), extraction (pull candidate facts like part numbers and concentrations), classification (decide what each fact is and which document class it belongs to), and reconciliation (check the facts against the compliance requirement). The automation decision is different at each stage. Which document-intelligence stages preserve traceability, and which risk destroying it? Ingestion and extraction preserve traceability when they record source coordinates and per-field confidence, because their outputs point back at something. Reconciliation carries the highest risk, since it is where an opaque generation step can substitute a plausible value for a sourced one. Traceability is a property of the link back to source, and it must survive every stage. How does document intelligence differ from generic OCR or a general-purpose generation model? OCR only converts pixels to characters — it has no concept of what a field means or whether it satisfies a rule, so it is a component, not the whole pipeline. A general-purpose generation model fails at the opposite pole: it produces fluent output including inferred values, collapsing extraction, classification, and reconciliation into one opaque step that kills the source link. Document intelligence uses models where they add lift while keeping the source anchor and verdict outside any generative step. Where does document intelligence stop being drafting assistance and start becoming compliance adjudication? Drafting assistance means the system proposes and a qualified human disposes. Adjudication means the system itself issues the pass/fail verdict that enters the record. The line is crossed the moment a reconciliation output is treated as authoritative without a human owning the logged verdict. How do you evaluate document-intelligence accuracy per field type and per document class? Break out extraction accuracy by field type and classification accuracy by document class rather than reporting one blended number, because a single figure hides the fields that matter most. Track traceability completeness as a hard gate, and treat reconciliation throughput and reduced manual-review time as the ROI signals the automation is buying. What has to be logged for a document-intelligence pipeline to remain auditable by an OEM reviewer? Every output fact needs its source document hash and region, the model and version that extracted it, the confidence score, the classification assigned, the reconciliation rule applied, and the identity of the human who owned the verdict. If any of these is missing, the audit trail has a gap a reviewer will surface later. Where this leaves a supplier-engineering team The same stage decomposition applies well beyond automotive: pharma regulatory-document automation faces the identical traceability problem, which is why AI document automation for pharma regulatory submissions leans on the same discipline of keeping source links intact through each stage. Regulated verticals differ in the rule set, not in the mechanism. If your team is weighing where to automate supplier-compliance document work, the productive question is not “which model should we use.” It is “at which stage does a fact stop pointing back at its source” — and that is exactly the drift a validation harness exists to monitor. Name the stage where traceability breaks, and you have named the risk the OEM reviewer will find first.
