You do not get consistent quality from many localisation and dubbing vendors by building custom tooling for each one. You get it by writing one QC standard first: pass/fail definitions, a shared error typology, severity levels, numeric thresholds and a sampling rule. You then reference that standard in every vendor contract and log every defect, whoever reports it, against the same taxonomy in one nonconformance record. Everything downstream (scorecards, alerts, vendor reviews) depends on those two decisions, and none of it can be fixed later in a dashboard if the standard underneath is vague.
How do we make sure localisation and dubbing vendors in every territory apply the same QC pass/fail criteria?
Vendors in every territory apply the same pass/fail criteria only when those criteria are written down, specific enough to give two reviewers the same result, and binding through the contract. A guide to multi-site quality control makes the same point in plainer terms: you cannot test against a standard that is not written down, and each checklist item should be specific enough that two different people checking the same thing reach the same result. Localisation has a particular version of this problem. Pedersen (2017) notes that in-house guidelines are the most common product-oriented quality tools many companies have, but since each company uses its own set, there is no way to compare them. Fifteen vendors with fifteen house styles produce fifteen incompatible quality signals.
What a QC acceptance criteria checklist for dubbing has to cover
Don’t write the checklist from scratch. Align it to published frameworks, which already give you the categories. Prime Video’s dubbing guidelines, for example, set out a unified framework for partners across three dimensions: translation and adaptation, voice casting and performance, and recording and mixing. Spiteri Miggiani’s TP model breaks dubbing errors into six categories (synchronisation, language, visuals and sound, translation, phonaesthetics and functionality) and 37 error specifics, each with a code for use during evaluation and feedback.
| Checklist area | What the standard should define | Where the vocabulary comes from |
|---|---|---|
| Synchronisation | Kinetic synchrony (gesture), labial synchrony (visible lip consonants such as m, b, p) and isochrony (utterance start and end) as separately scored checks. | Prime Video dubbing guidelines |
| Sync defects in delivered assets | Coded defect names such as “Dialogue Out of Sync” and “Effects Out of Sync” for audio, and global drift early, global drift late, global offset and intermittent sync for timed text. | Netflix asset and timed-text QC error codes |
| Loudness and level | Dubbed audio levels that match the original version, plus automated loudness tests. | Prime Video guidelines; DPP UK QC requirements |
| Manual “eyeball” checks | Named checks such as audio sync, buzzing and unclear sound, done before delivery. | DPP UK QC requirements |
| Translation accuracy | Mistranslation, unnecessary omission or addition, unnecessary loss and improper translation. | TP model error definitions |
| Script functionality | Dubbing notations, character allocation, time codes and compliance with guidelines. | TP model functionality category |
| Sign-off | Who signs off QC, and a QC report delivered with every file. | DPP UK QC requirements |
For the technical tests, the EBU publishes an open catalogue of hundreds of QC test definitions that tools can implement and content owners can cite in delivery specifications. The DPP built its UK broadcaster requirements by taking those EBU definitions and setting a minimum set of tests and tolerance levels. We would follow the same pattern: cite the shared definition, then state your own tolerance next to it.
You will notice the table gives categories and no millisecond values. The closest precedent is subtitling: Pedersen’s FAR model treats a spotting error under one second as minor and one that is out of sync by more than an utterance as serious. Your sync tolerance in milliseconds and your loudness target are choices you make per content type and write into the standard as numbers. They are not facts you can look up. Vague standards, the multi-site guide warns, produce inconsistent results.
Severity classes and a numeric pass/fail line
Severity classes are what let a reviewer in one territory and a reviewer in another reach the same verdict on different languages. MQM uses four default severity levels (neutral, minor, major, critical) with penalty multipliers of 0, 1, 5 and 25. The MQM Council recommends that the gap between levels stays exponential to reflect rising risk, while letting implementers set their own values. Pass/fail then comes from a tolerance: Acceptable Penalty Points (APP) per reference word count, usually 1,000 words, tied to a passing threshold. Smartling’s LQA tooling, for one, defaults APP to 20, which maps to a raw quality score threshold of 98 and a calibrated threshold of 80.
The TAUS Dynamic Quality Framework treats the pass/fail threshold as something that varies by content type and target language and is set at project creation. That flexibility is fine as long as the standard states which threshold applies to which content class, so that “flexible” never quietly becomes “negotiated per vendor”.
Setting the sampling rate without reviewing everything
The sampling rate is a trade between statistical confidence and review cost, and the standard should state it per defect class instead of leaving it as one global percentage. ISO 2859-1 classifies nonconformities by seriousness, gives the most serious class a very small acceptance quality limit (AQL), and allows every item to be inspected for designated critical classes. The same standard notes that the probability of acceptance at a given AQL depends on sample size and is generally higher for large samples than for small ones. On the translation side, the multi-range MQM paper (arXiv, 2024) reports that traditional localisation typically samples medium sizes of 500 to 5,000 words, and argues that small, medium and large samples each need a different evaluation approach. With dozens of territories, we would sample critical classes exhaustively or near it and sample the rest at a rate the review budget can actually sustain.
Writing the standard into contracts without breaking live work
A QC standard only binds vendors once the contract references it. Standards bodies are built for exactly that: Rivers (2018) describes an ASTM Standard Practice as something that can be referenced in a contract and audited by either party. ISO 2859-1 likewise requires the AQL to be designated in the contract or by the responsible authority. Rivers also notes that exceptions to a standard must be documented and justified, with personnel qualifications in low-density languages given as the example. That mechanism is useful for languages where the full standard cannot yet be met.
On phasing, we would introduce the standard at each contract’s renewal, or give vendors a transition grace period with scores recorded but not penalised. Switching criteria mid-contract on work already in production invites disputes and resistance. Before enforcement, replay past evaluations against the new scoring model, which is how one MQM deployment described in the arXiv paper chose its weights and thresholds: it worked out the error level that historically separated unacceptable from acceptable translations, then tested candidate multipliers against live work.
A vendor and territory scorecard that shows who is drifting
Once every vendor is scored against the same standard, a scorecard can show which vendors or territories produce the most flagged defects over time. The TAUS Quality Dashboard was designed so that project, vendor and quality managers could track and benchmark translation quality, with filters for language pair, time span, project and content type, and reports on both the number and the type of errors. Smartling’s error density report breaks error counts down by project, language and job, and Smartling positions it for reviewing vendor agreements and SLAs. The scorecard is a remediation tool, not a league table. A failed check needs a corrective action with an owner and a deadline, tracked to completion.
One nonconformance tracking system for every vendor’s defects
The system that lets you log, categorise and compare QC nonconformances from all your vendors in one place is a centralised nonconformance tracker, with one governed defect taxonomy and a normalisation step before any cross-vendor comparison. Per-vendor spreadsheets cannot do this. The vocabulary comes from quality management: ISO 9000 defines a nonconformity as “non-fulfillment of a requirement”, and ISO’s auditing guidance recommends recording nonconformities systematically on a Nonconformity Report (NCR) for traceability. A well-documented nonconformity has three parts: the evidence, a record of the requirement it breaches, and a statement of the nonconformity. For localisation, “the requirement” is the clause of your written QC standard. That is why the standard has to exist before the tracker.
Taxonomy discipline matters more than the software. A nonconformance software selection guide (Assyro) warns that if one site codes a defect as “machining” and another codes the same failure as “tooling”, cross-site trend analysis produces noise. It recommends a centrally governed taxonomy with restricted free-text categorisation. Multi-site organisations, the same guide says, need a shared core taxonomy with clearly bounded local extensions. For vendors, the TP model codes or MQM error types become that shared core, and any territory-specific checks sit as extensions underneath.
Can vendor-reported defects and our own spot-check audits live in the same tracker?
Yes, and they should, as long as every record carries its origin (vendor self-report or internal audit) as a field. Supplier-originated nonconformances need their own workflow, because evidence ownership is split and responsibility may be disputed. The Assyro guide recommends a supplier portal where suppliers view the NCR, respond and attach evidence, with the record keeping both parties’ positions and the final determination. Translation tooling has a parallel in arbitration: in Smartling LQA, translators can dispute recorded errors and reviewers accept or reject the dispute. Keeping the two data sources side by side also shows you where a vendor under-reports. Self-reported defect rates that sit consistently below audited rates for the same vendor are a finding in themselves.
Normalising defect rates before comparing vendors
Raw defect counts reward vendors that handle less content, so rates have to be normalised by volume before any ranking. MQM normalises to a reference word count, usually 1,000 words, and the FAR model divides penalty scores by the number of subtitles.
Worked example: what an MQM tolerance means per 1,000 words. The MQM Council’s example sets the tolerance at 10 minor errors or 2 major errors per 1,000 words, which gives a raw quality score of 99. With the default multipliers (minor 1, major 5), both limits equal the same budget of 10 penalty points: 10 × 1 and 2 × 5. To compare vendors, convert each vendor’s penalty total to points per 1,000 evaluated words, using the same weights for everyone.
Two cautions apply. First, MQM calibrated scores may not be comparable across tasks or organisations, and the MQM Council suggests non-calibrated scores where direct comparability matters. Use raw, normalised penalties for the cross-vendor view, even if each vendor’s pass/fail report uses calibrated scores. Second, choose the normalisation unit deliberately. Pick one unit per content type, such as per 1,000 words, per programme hour or per asset, write it into the standard and leave it alone.
Configure a supplier-quality platform or build something custom?
For most content operations, configuring an existing supplier-quality or nonconformance platform is the right starting point. A custom build is worth considering mainly when evaluation shows no platform can group results by territory, or the vendor data cannot be fitted into one common intake schema. Supplier quality management products already handle nonconformance, CAPA and audit workflows. A supplier quality software comparison (Gitnux) describes ComplianceQuest, ETQ and AssurX handling supplier nonconformance and corrective and preventive action (CAPA) workflows, with ComplianceQuest and AssurX also covering audits. It also describes GEP adding rule-driven notifications and status transitions. Bear in mind that the comparison describes these tools for procurement, quality and manufacturing teams. The sources describe nothing localisation-specific, so a vendor and territory hierarchy is something to test during evaluation, not assume.
| Situation | Choice |
|---|---|
| You need NCR tracking, owners and closure only | A standalone NCR tool |
| You also need CAPA and document control linked to each NCR | A full QMS platform |
| The platform cannot group or report by territory and vendor | Configure reporting on exported data, or build that layer only |
| Vendor formats cannot fit one intake template even after mapping | Custom intake, with the tracker itself still off the shelf |
The same Gitnux comparison warns that these systems fail when workflow definitions are inconsistent across sites or evidence capture is under-specified. That is the written-standard problem again, showing up in the tool.
Rolling reject rates and threshold alerts per vendor and territory
Whether a platform computes rolling reject rates per vendor and per territory is something to verify during evaluation; the defensible way to set the alert line is a proportion control chart, not a round number. The NIST/SEMATECH handbook gives the p-chart upper control limit as p + 3√(p(1−p)/n), where p is the expected proportion nonconforming and n the sample size. Because n sits in the denominator, a territory with few reviewed assets gets a wider limit than a high-volume one. A single fixed threshold ignores that difference. Alerts need enough history per vendor and territory to estimate p before they mean anything. How a breached threshold turns into escalation and corrective action is a policy question in its own right; our article on setting cost-weighted thresholds and escalation triggers works through that design in the monitoring setting.
Getting vendors with uneven tooling to feed one system
Vendors with different levels of tooling maturity can feed one system through a lightweight, templated intake (a common schema delivered as CSV or through an API) rather than point-to-point integrations. For text assets, a standard already exists: XLIFF 2.0 is an interchange format intended to be understood by any localisation provider using a conformant tool. Low-maturity vendors may need a manual or batch upload route, which is acceptable as long as you know the data arrives later and your alert windows allow for it.
Routing only low-confidence dubbed tracks to human reviewers
A human-in-the-loop QC pass scales across vendors when an automated confidence score decides which tracks go to human review. Tracks below a chosen threshold go to a regional localisation lead, and the rest pass automatically under one review protocol, whoever delivered them. TechnoLynx can build the confidence scoring and routing for this, tune the escalation cutoff against human-reviewed tracks and wire the gate into delivery, as part of its AI engineering for media, broadcast and telecom.
Smartling describes the same shift for translation: quality estimation scores every segment so review effort concentrates on flagged, low-confidence content, which Smartling calls review by exception. Smartling also separates automated quality checks, which catch objective rule-based errors, from LQA, where human linguists judge issues that need human judgement.
The escalation threshold is a cost and risk decision, not a technical constant. In a 2026 industrial defect-classification study (RobustDefect-LLM), a deliberately conservative policy (confidence of at least 0.90) sent 87.78% of samples to human review and automated only 12.22%. On that test set it let no classification error through to an automated action. That is not dubbing data, but it shows the trade clearly: a cautious threshold buys safety with reviewer hours. We would set the cutoff where the cost of a defect reaching release balances reviewer capacity, then revisit it whenever volume or vendor mix changes.
Confidence is a proxy, and it misses things. Improper translation in the TP model is judged against target-culture sensitivity and viewer expectations. That kind of error may not register in a score at all, so a share of high-confidence tracks should still be spot-checked at random. Vendor delivery formats can also bias scores. Normalise formats before scoring, and recalibrate per language or vendor if scores show systematic offsets.
Calibration across languages is the part no tool settles
Every mechanism above assumes that a “major” error in one language means the same thing as a “major” error in another. Keeping that equivalence is the reviewers’ job, not the software’s. A TAUS DQF paper (2014) notes that adding evaluators and giving them instructions beforehand improves the credibility of evaluation results. In practice, that means periodic calibration sessions where reviewers from different territories score the same samples and reconcile differences before those differences reach the scorecard.
