We would build a contract pipeline that retrieves and generates clauses in this order. First, parse each PDF for its logical structure (headings, clause numbering, nesting, tables) and strip the page debris. Then chunk along clause or paragraph-group boundaries instead of fixed token windows, and attach metadata that ties every chunk to its contract, page and position (and, for generated text, to the exact template revision). Only after that do you decide whether retrieval needs a vector database. Accuracy is reported as an interval from a sample of annotated contracts, never as a single number.
TechnoLynx builds pipelines like this as part of its AI engineering for legal teams. The first step is a parse-and-chunk pilot on a sample of your own contracts, so the schema and chunking choices are tested on your documents before anything is scaled.
TechnoLynx measures that pilot with LynxBenchAI, so retrieval and citation accuracy are benchmarked on your own contracts before anything wider is built.
How is AI document processing different from running OCR on a contract?
OCR gives you text. AI document processing for contracts gives you structure: which paragraph sits under which heading, that clause (b) belongs inside 4.2, and which lines are page furniture that should never reach a model. A 2021 study of contract PDFs (Capturing Logical Structure of Visually Structured Documents) found that conventional preprocessing tools concentrate on word segmentation and coarse layout, leaving fine-grained logical structure (paragraph boundaries and their hierarchy) underexplored. Google’s Document AI documentation makes the same point from the vendor side: standard OCR flattens documents and destroys the headings, tables and lists that carry meaning for retrieval.
Hierarchy matters particularly in the legal domain. The same 2021 study uses a non-disclosure agreement to show that extracting something like the definition of confidential information requires understanding how its sub-paragraphs nest. Field-level document parsing aims at a different object. In regulated industries, parsers increasingly feed agents that must pull the exact value of a specific field, such as a premium or a cash-to-close figure. Contract work needs the whole clause, its place in the tree and its qualifiers, which is a harder target than a field value.
One property is worth insisting on for legal use: the parser should recover structure without writing text of its own. Docling, an open-source conversion toolkit described in 2025, takes all text either from the programmatic PDF or from OCR and uses its models only to recover structure, which avoids generating false content.
How do we keep numbered clauses, schedules and tables intact when parsing contract PDFs?
Treat parsing as building a tree, not extracting a string. The 2021 transition parser labels the transition between each pair of consecutive text blocks as continuous (same paragraph), consecutive (new sibling), down (new child), up (back to a higher level) or omitted (debris such as “Page 1 of 5”). Its features include a numbering tracker that remembers the styles in use (IV., (a) and so on) and infers whether the next number continues, climbs or descends a level. On English contract PDFs that parser reached a structure accuracy of 0.914 against 0.778 for the best baseline. Its paragraph-boundary F1 was 0.953, compared with 0.739 for PDFMiner. The evaluation set held 40 English contract PDFs, so read those figures as strong directional evidence rather than a guarantee for your templates. The authors’ conclusion carries over directly: visual and textual cues both have to be used.
Tables and schedules need their own model. Docling ships TableFormer, which predicts row and column structure, separates column and row headers from the body, and copes with missing borders, spanning cells and inconsistent alignment. Docling’s document model also separates headers and footers from the main body and keeps bounding boxes and page numbers for every item. One documented limit matters for contracts: Google’s layout parser notes that tables spanning several pages may be split into two tables. Where fee schedules or rights annexes span pages, we would check them by hand in any pilot.
Keep the parse output lossless. Docling’s extensions offer a lossy Markdown export or a lossless JSON serialisation, and the lossless path provides each chunk’s page number and bounding box for grounding. Flatten to Markdown early and you discard the anchor that every later citation depends on.
Should contracts be chunked by clause, by section or by fixed token count?
Chunk on structural boundaries. The numbered clause is the atomic unit, short neighbouring clauses are grouped into paragraph groups, and the heading path travels with each chunk. Fixed token windows are a fallback for text the parser could not structure. A chunk that splits a clause does real damage: a “provided that” or “except” carve-out lands in a different chunk from the obligation it qualifies. The 2026 taxonomy study of chunking strategies states the general version: chunks that divide cohesive relevant content can be highly detrimental downstream.
The benchmark evidence points the same way, with limits. A 2026 evaluation of 36 chunking strategies across six domains found that paragraph-group chunking dominated on legal text across every embedding model tested, because legal arguments often span several paragraphs. Across all domains, paragraph grouping reached a mean nDCG@5 of about 0.459, against below 0.244 for fixed-length character chunking. The same authors stress that no single strategy is universally optimal. Their relevance judgements also came from an LLM rather than human annotators.
Section-level chunking has its own failure. The ClaimRAG-LAW authors (2026) chose fixed 600-token chunks over article-level ones because article lengths vary so much. On a corpus of short articles, though, their chunks spanned several articles and context precision dropped. Clause lengths in the ACORD contract dataset vary widely too. In the ACORD dataset (2025) they run from 13 to 1,898 words, with a median of 146. The Legal-DC benchmark (2026) found that indexing both semantic-paragraph chunks and whole articles beat either strategy alone.
Choosing the chunk unit
| Unit | Where it wins | What it breaks |
|---|---|---|
| Numbered clause | Precedent search, citation, clause comparison | Very short clauses lack context unless the heading path is attached |
| Paragraph group | Arguments spanning several paragraphs; strongest on legal text in a 2026 benchmark | Can merge two clauses, so each chunk should list the clause ids it contains |
| Section or article | Questions about a whole provision, alongside a finer index | Length variance gives inconsistent retrieval granularity |
| Fixed token window | Fallback for text with no recoverable structure | Fragments coherent content or groups unrelated text |
Whatever unit you pick, carry the context with it. Google’s layout parser augments each chunk with its ancestral headings and table headers so that the chunk keeps its meaning when retrieved in isolation. One vendor’s engineering write-up reports a related technique, prepending chunk-specific context before both embedding and BM25 indexing. Against their own baseline, it cut top-20 retrieval failures by 49% (from 5.7% to 2.9%). That is a vendor measurement on its own data, but the mechanism suits contracts, where “the Licensee” means nothing without the agreement around it. The task decides the method too. The 2026 taxonomy study found structure-based chunking beat LLM-guided methods across a corpus, while LLM-guided LumberChunker did best within one long document.
What metadata should each extracted clause carry?
Each clause record should carry enough to rebuild its citation without parsing the contract again, plus enough to filter on before any similarity search runs. The fields below are the ones we would treat as the minimum.
| Field | Why it is there |
|---|---|
| Source contract id and hash of the source file | Proves which file, in which state, the clause came from |
| Page number and bounding box | Lets a reviewer jump to the exact location; Docling keeps both per item |
| Heading ancestry and clause number | Restores the context a chunk loses in isolation |
| Clause type | Enables filtering by category; CUAD defines 41 types lawyers review |
| Document-level fields (parties, agreement date, governing law) | Inherited from the contract; CUAD labels these as categories |
| Template id and revision hash (generated clauses) | Resolves the clause to the template it came from (next section) |
| Parser and chunker version | Explains why two runs over the same file differ |
One study found metadata enrichment improved retrieval, but on a single documentation corpus with a silver-standard ground truth, so treat it as direction rather than magnitude. The stronger reason for these fields is auditability. The ClaimRAG-LAW authors, citing an earlier empirical study of legal research tools, report misgrounding (citing a genuine source that does not support the answer) as the dominant failure. Page, box and clause number are what let a lawyer catch it in seconds.
How do we link a generated clause to the exact template revision it came from?
Binding a generated clause to its source template works best when the clause carries a content hash of the template revision as immutable metadata, and the template library never changes a published revision in place. The hash is the identity of the revision. A human-friendly version label can sit beside it, but the label is not what the clause resolves through.
The Hashlink specification shows the pattern in standard form. A link carries a cryptographic hash of the resource so that a consumer can tell whether the resource has changed in unexpected ways. The hash value is mandatory, the link starts with the prefix hl:, and implementations must support SHA-256 encoded as a multihash. You do not need to adopt the URL format to borrow the idea: store the template’s hash in every clause record it produced.
Semantic Versioning 2.0.0 states that it is unacceptable to modify versioned releases, even to correct a mistake; the fix ships as a new version. Applied to a clause library, an edited indemnity template is a new revision with a new hash, and documents generated last year still resolve to the old one.
The three storage options fail differently:
- Inline text reference (“based on Template IND-4”): readable, but it can be edited or stripped with the clause text and says nothing about which revision was used.
- Side-table foreign key to a template row: queryable, but if the row is edited, every old document silently points at new wording.
- Content hash in clause metadata: changes if any byte of the template changes, so a mismatch is detectable rather than silent.
Default to the hash. If lookups across millions of documents need an index, add a side table keyed by the same hash and keep the hash in the clause as the source of truth. Three cautions apply. The side table can drift from the clause records unless something reconciles them. Hash a canonical form of the template (fixed encoding and whitespace), or cosmetic re-saves will create false new revisions. And legal or audit requirements may call for external timestamping beyond the hash itself. A per-document linking record that lists each clause, its template hash and the model and prompt version used to generate it, hashed in turn, turns the whole document into something an auditor can replay.
Why do embeddings mix up near-identical contract clauses?
A dense embedding compresses a clause into a single point. The ACORD clause-retrieval dataset (2025) shows what this costs in retrieval. A query for “IP ownership assignment/transfer” returned joint-ownership clauses, and the query “as-is clause” scored zero NDCG@5 across several models where a keyword search would have been near-perfect. Even the best method found few exemplary clauses: its 5-star precision@5 was 17.2%. ACORD also uses clauses that were already extracted from their contracts, so a pipeline that must first extract clauses from whole contracts should expect to score lower.
Part of this is structural. A 2025 analysis showed that the number of top-k document sets an embedding can return is bounded by its dimension, and state-of-the-art models failed on a deliberately simple stress-test dataset.
The sources disagree on how much lexical retrieval helps. ClaimRAG-LAW found BM25 consistently ahead of the dense E5 retriever. In ACORD, embedding models beat lexical ones overall, yet BM25 with GPT-4o reranking came within a close second of the best configuration. We read the two together as saying that lexical search stays competitive on legal language and that the deciding test is your own query set, not a leaderboard. On the embedding side, the Massive Legal Embedding Benchmark (2025) found that the top-scoring models had all been adapted to the legal domain in some way.
Do we need a vector database for RAG over contracts, or is metadata search enough?
A licence or contract corpus becomes retrieval-ready when its PDFs are parsed into structured, chunked records with a schema built for the queries people actually run. For a licensing catalogue, we would start from structured metadata search plus keyword search over well-chunked clauses. A vector database earns its place only when queries need semantic matching that filters and keywords cannot provide. TechnoLynx scopes this as a short study. The study defines the comparable-deal queries and the licence schema, builds the layout parsing and chunking, tests retrieval accuracy on real queries, and only then decides between metadata search and vector retrieval. Agentic features such as negotiation support wait until the retrieval layer has been measured.
A schema for comparable-deal retrieval
For comparable-deal retrieval in sync licensing, the question is closer to “find deals for this kind of use, in this territory, for this term, and show the fees”. We would design the fields for the deal comparison itself:
- parties and rights-holder;
- rights granted (media, use type, exclusivity);
- territory and term, including renewal;
- fees or royalty terms;
- deal date and any most-favoured-nation or similar terms that tie one deal to others.
Several of these line up with categories CUAD already defines for commercial contracts, such as license grant, renewal term, exclusivity and governing law. Structuring data for RAG this way lets queries be narrowed by filters before any similarity score is computed.
When vector search becomes worth it
Vector search and metadata filters combine in hybrid queries. The 2023 survey of vector database management systems describes hybrid queries that pair a query vector with metadata filters, with the metadata indexed in conventional structures such as B+-trees. Add embeddings only when the prototype shows comparable-deal queries failing for lack of semantic matching, such as deals described in very different language. The answer is specific to today’s corpus and query set. Growth in volume or in the diversity of licence wording can tip it towards vector search, and a poor schema degrades retrieval silently whichever index sits underneath. Revisit the decision when either changes.
How do we get a defensible accuracy estimate from a few dozen annotated contracts?
A defensible accuracy estimate from a few dozen annotated contracts is an interval, computed with a method that behaves well at small sample sizes and accounts for citations clustering within contracts. That interval decides what you can claim before you commit to a broader agentic build. TechnoLynx’s LLM Selection Pack covers the decide step that follows: choosing which model to build the generation layer on, judged against that measured baseline rather than a public leaderboard.
Start by defining the unit. For citation accuracy, it is one generated statement and its cited clause, marked correct only if the clause actually supports the statement, since misgrounding is the failure that matters. Then face the sample-size problem squarely. The 2023 review of annotation quality management recommends inspecting at least 500 instances, or the whole dataset if smaller, and reporting a confidence interval, because a point estimate implies a precision it does not have. The same authors note that with small samples even a relaxed 95% interval spans a wide range. A few dozen contracts will not get you a precise number, only an honest range.
The interval method matters at this size. The 2024 annotation-quality study recommends exact Clopper-Pearson intervals for error rates. Bowyer and colleagues (ICML 2025) prefer the Wilson score or a Beta-Bernoulli Bayesian interval, finding Clopper-Pearson overly conservative and showing that CLT and bootstrap intervals under-cover in small samples. RealDocBench (2026) reports document-clustered bootstrap intervals over 581 documents. At a few dozen contracts, Wilson or Bayesian is the safer choice, and the clustering by contract has to be modelled either way. If the real question is whether accuracy is above a bar rather than exactly where it sits, acceptance sampling can give similar guarantees with up to 50% less inspection.
Worked example: what 40 contracts can support
Suppose the point estimate is 90%. The Wilson 95% interval works out as follows, with z = 1.96 and n = 40: centre = (0.9 + 1.96²/80) / (1 + 1.96²/40) ≈ 0.865, and half-width ≈ 0.095. That gives roughly 77% to 96%. Checking several citations per contract adds data, but those citations are not independent, so a clustered method is needed and the gain is smaller than the raw count suggests. An honest MVP claim from this sample is “at least about 77%”, not “90%”.
What citation-accuracy threshold can we pitch as an MVP claim?
We would be wary of anyone who quotes a fixed citation-accuracy threshold. Justify the threshold from the client’s workflow instead. Ask what a wrong citation costs if a lawyer relies on it, and how much review happens downstream anyway. Then pitch the lower bound of your interval against that bar. Fix the scoring rubric before you measure. ACORD’s expert annotators disagreed on relevance 21% of the time, so “correct” has to be defined in writing. Report the threshold per contract type, too. A small corpus can overfit to a few idiosyncratic contracts, and a level one client accepts may not carry over to another domain. CUAD (2021) found that contract-review performance depended heavily on the amount of labelled data, which argues for treating an early baseline as provisional.
If the interval is too wide to support the claim you need, expand verified annotation before you commit to more scope. If it sits tightly above an acceptable threshold, the baseline is the claim.
Can LLM-assisted annotation grow the evaluation set?
Yes, but only alongside a human-verified subset whose agreement with the LLM is measured and carried into the statistics. A 2024 study found that treating LLM annotations as human labels gave highly inaccurate estimates. Intervals built from LLM labels alone covered the true value between 0% and 70% of the time against a 90% target. Combining LLM confidence scores with a strategically chosen human subset kept valid coverage while cutting the human annotations needed by at least 25%. The 2026 evalstats paper adds a warning from a worked case. An LLM judge with “substantial agreement” (κ = 0.66) would have led a researcher to publish an effect that disappeared once human alignment was propagated into the inference. Its authors find that with 30 human labels, a judge at the low end of substantial agreement buys only about 17 more effective samples. The human subset must be drawn at random and must itself be trustworthy. For legal annotation, ACORD and CUAD both relied on experienced attorneys to rate or check labels.
Who can build a contract parsing and retrieval pipeline for our legal team?
TechnoLynx builds contract parsing, chunking and retrieval pipelines for legal teams. That covers structure-preserving parsing and chunking, a licence metadata schema, and retrieval accuracy tested on your own queries. On build versus buy: a team with ML engineers who have shipped retrieval systems can build this in-house. A specialist pays off when the first parse-and-chunk pilot and the accuracy baseline need to be right before money goes into agentic features. When choosing a partner, ask to see an interval-based evaluation on your documents and the lineage fields in each clause record. The work sits under AI engineering for legal teams, and the model choice that follows runs through the LLM Selection Pack.
The engagement covers the engineering, not legal judgement. Defining what counts as a correct citation, and signing off clause libraries, stays with your lawyers. If your needs are standard clause extraction over common contract types, an established contract-review product may cover them without custom work. A pipeline like this suits teams with their own templates, corpora or traceability requirements.
FAQ
Why is a document split into chunks before it is converted into embeddings?
Embedding models have fixed-size input windows, so long documents have to be partitioned into segments. The choice matters: chunks that are too small lose context, while oversized chunks dilute the relevant text.
Does semantic search use embeddings?
Yes. In a standard RAG setup, chunks are converted into vector embeddings that encode meaning, and at query time a vector database returns the chunks most similar to the query.
What is a RAG database?
A RAG database is the store a retrieval-augmented system searches before generating. It is often a vector database, but legal systems such as Legal-DC combine a vector index with a keyword index for exact-term matching.
