A defensible gauge R and R study for a vision inspection system starts by picking the study type that matches what the gauge outputs: a variable study for a measured dimension, or attribute agreement for a pass/fail call. It then builds part colour and fixture wear into the design as sources of variation, and the acceptance criteria are written down before any data is collected. Auditors are not checking one universal threshold. They check that the measurement system was studied with methods and criteria from the MSA reference manuals, or with alternatives the customer has approved, and that the reported numbers fit the documented design.
The running case here: a vision gauge is replacing a manual micrometer on a foam part approved in three colours, and the lighting was tuned on the grey sample. Every choice below follows from that situation.
Gauge R and R for a vision gauge: pick the study type before anything else
A Gage R&R study estimates how much of the observed variation comes from short-term repeatability and how much from changing measurement conditions, such as different appraisers. The structure of the study depends on whether parts can be remeasured unchanged and whether the output is a number or a category.
| Gauge output and situation | Study design | What it estimates | Design figures from the sources |
|---|---|---|---|
| Continuous value (a dimension or score) on parts every appraiser can remeasure unchanged | Crossed variable GR&R | Part, appraiser, repeatability and interaction effects | 3 measurements × at least 10 parts × 3 operators, per Vision Systems Design |
| Measurement changes or destroys the item | Nested GR&R | Variation with items nested within appraisers or conditions | Analysed to match the nesting |
| Extra factors matter (fixtures, shifts, gauges, part colours) | Expanded GR&R | Additional factors and their interactions with operator and part | Three or more factors, unbalanced data allowed |
| Pass/fail or categorical call | Attribute agreement analysis | Agreement within appraisers, between appraisers and against a reference | At least 50 samples and at least 3 appraisers, per Minitab |
Each output type has its own statistics: Gauge R&R for a measured dimension, attribute agreement analysis for an accept/reject call. How to compute the variance components themselves (ANOVA versus average-and-range) is a separate topic. The one rule that matters here is that the analysis method should fit the design, not whatever the software uses by default.
Two steps come before the GR&R. The system has to be calibrated against artefacts traceable to national standards, so that measurements are in real units and not pixels, with lens distortion corrected first. Then a Type I study (one operator, at least 30 inspections of a single calibration artefact per attribute) measures bias and gauge capability, before reproducibility is tested at all.
Does the lighting give the same defect visibility on every part colour, or only on the grey sample?
You cannot assume it does. Lighting tuned on the grey part supports claims about the grey part only, and the other two colours stay unverified until they are tested under the same conditions.
TechnoLynx can run this colour robustness audit on your own camera, optics and lighting. TechnoLynx starts by setting contrast and defect-visibility acceptance criteria for each colour. TechnoLynx then tests lighting variation crossed with every approved colour and reports whether one threshold meets the criteria, with false-reject and false-accept rates for each option. This is part of TechnoLynx’s AI engineering for manufacturing and automotive work.
The reason lies in how materials interact with light. Materials reflect and absorb wavelengths differently, and that difference is what illumination uses to create contrast between a feature and its background. Under visible light, a change of pigment can change that contrast. Cognex states that good machine-vision lighting must be consistent, so that normal variation in parts does not change the contrast in the image. A practical lighting guide from Advanced Illumination lists robustness as one of three acceptance criteria for a lighting solution, alongside maximising contrast on the features of interest and minimising it elsewhere. An article in Quality Magazine says plainly that a lighting approach must be tested carefully, however it was engineered.
Whether your three colours behave differently is an empirical question that only an audit answers. We would set the audit up like this: put known-defect parts in every colour, vary the lighting within the range the line will actually see, record true-positive and false-negative rates per colour, and compare each colour with the grey baseline. How to run a low-cost pilot that audits perception robustness across lighting and sensor variation is covered separately.
If the audit does find a gap, wavelength is one lever. Near-infrared reflectivity depends more on material texture and composition than on pigment, so NIR illumination can even out contrast differences that come from visible colour. That helps only when the defect itself is not a colour difference, so treat it as a design change with its own evidence.
One global threshold, or a threshold per colour?
If contrast and defect visibility are equivalent across all three colours within the tolerance you set beforehand, keep one global threshold. If one colour shows a consistent gap beyond that tolerance, add per-colour threshold overrides for that colour rather than retuning the whole system, which would disturb the colours that already pass.
Each override is configuration that has to be version-controlled together with the logic that identifies the part’s colour, and a wrong colour call then selects the wrong threshold. In the Gauge R&R, colour becomes a factor of the study whichever way you decide, because the study has to show that the chosen thresholding holds on every colour.
What sampling plan covers every colour and the fixture’s full wear life?
A statistically defensible plan crosses the fixture’s full wear range with all three approved colours, uses reference parts with known defect states wherever they can be made, and is judged against acceptance criteria fixed before the first measurement. If a study leaves out one colour or one wear interval, it cannot support a claim of coverage across the approved range.
Parts should span the range the measurement system will meet in use, including the finishes and conditions that could affect measurement performance, and they should not be chosen to produce a favourable percentage. Fixture wear belongs in the design because repeatability covers more than the camera. The repeatability component can include fixture, positioning and procedural effects that stay inside the “same-condition” repeat, and worn instruments or fixtures are a known cause of measurement error. A study run only on a new fixture describes the system on day one and nothing after.
The structure that fits is an expanded GR&R. It adds factors beyond operator and part, can estimate their interactions with operator and part, and tolerates unbalanced data. Colour and wear stage are chosen on purpose, not sampled at random, so they are fixed factors. Minitab notes that treating fixed factors as random can over- or under-emphasise how much they matter.
Worked example: sizing a colour-by-wear expanded study
Assumptions, stated as assumptions: three approved colours, and fixture wear sampled at three points (new, mid-life, end of life). That gives 3 × 3 = 9 colour-wear cells. In Minitab’s expanded example, the number of parts was cut from 10 to 5 to keep the study manageable. In that example, a full ten-part study for each of three gages would have needed 180 measurements, which Minitab called unacceptably large.
Cutting parts has a statistical cost. With fewer than 10 parts, Minitab strongly recommends entering a historical process standard deviation and judging the gauge on %Process rather than %Study Variation. Even the conventional design is imprecise. Minitab’s simulation work found that 10 parts, 3 operators and 2 replicates estimate part-to-part variation with a margin of error of 35% to 40% at 90% confidence (45% at 95%), and that about 35 parts are needed for 90% confidence of being within 20% of the true value. The same work found the typical design adequate for repeatability, with about 30 degrees of freedom for error. Precision is weakest on the part-to-part side, and that side drives the accept decision.
Reference parts with known defects, or production parts re-measured by appraisers?
Use reference parts with known defect states at each wear stage and in each colour wherever you can produce them. Production parts with repeated appraiser measurement show precision, not bias. They cannot tell you whether the gauge is right. A reference value is the known, correct rating for a standard sample, used to assess whether appraisers rate correctly. One MSA source puts the risk directly: results can be repeatable, reproducible and still wrong, so bias has to be measured alongside GR&R. If systematic bias comes from colour or wear, a study built on repeated measurement without reference values will not see it.
ISPE’s guidance on visual inspection accepts defective units taken from production rejects or equivalent defects re-created in a lab, standardised by expert inspectors and with their condition hidden from the people being assessed. For pass/fail studies, Minitab advises including several marginally acceptable and marginally unacceptable samples. One customer guideline requires attribute studies to draw from the entire tolerance range, with 20% of pieces out of specification. Building a known-defect challenge set for vision inspection qualification is covered separately.
If reference parts cannot be made at every wear stage, fall back to production parts with repeated measurement. Say in the protocol that the study bounds precision but not accuracy, and set the acceptance limits more conservatively to match.
Before the micrometer backup is withdrawn
Withdrawing the manual gauge needs one more piece of evidence beyond the gauge’s own GR&R: a method-comparison study on paired samples, ideally with seeded known defects, showing the new method is at least as capable as the old one. That comparison needs a documented picture of the old method, including any existing Gauge R&R on the manual gauge. Do not assume the manual baseline is good. In a 2022 study in Safety, 50 practitioners each judged photographs of 26 aero-engine blades twice, and their inspection accuracy averaged 67.7%. This evidence set is a validation package of the kind production AI reliability work produces before release. The full path from study results to sign-off is covered in proving the vision system is good enough to stand alone.
Gage R&R acceptance criteria an auditor will accept for a vision system
No single %R&R, ndc or kappa value is accepted by every auditor. IATF 16949 clause 7.1.5.1.1 requires analytical methods and acceptance criteria that conform to the reference manuals on measurement systems analysis, allows alternatives if the customer approves them, and requires records of that approval to be kept. Within that frame, acceptability depends on how the measurement will be used, the process-to-tolerance relationship, the study design and the cost of a wrong decision. Percentage bands are rules of thumb, not automatic pass/fail lines. The defensible approach is to choose the criteria for the gauge’s intended use, write down why, and define them in advance.
The conventional starting points, with the basis each one applies to:
| Statistic | Applies to | Acceptable | Marginal | Unacceptable |
|---|---|---|---|---|
| Gauge R&R as % of study variation or % of tolerance | Variable studies | Under 10% | 10% to 30% | Over 30% |
| Gauge R&R % contribution (variance-based) | Variable studies | Under 1% | 1% to 9% | Over 9% |
| Number of distinct categories (ndc) | Variable studies | 5 or more | Under 2: no value for process control | |
| Kappa | Attribute studies | 0.75 or more; 0.90 preferred for critical characteristics | 0.40 to 0.75 | Below 0.40 |
| Effectiveness | Attribute studies | 90% or more | 80% to 90% | Below 80% |
| Miss rate (bad parts passed) | Attribute studies | 2% or less | 2% to 5% | Above 5% |
| False alarm rate (good parts rejected) | Attribute studies | 5% or less | 5% to 10% | Above 10% |
The column you read is the first thing to check on any GR&R output. The 10/30 bands come from the AIAG MSA reference manual (4th edition, 2010) and apply to GR&R as a percentage of study variation or of tolerance, not to percent contribution, and software prints all of these side by side. Study variation depends on the parts chosen: hand-picked parts that span the tolerance flatter the gauge, whereas percent of tolerance does not move with the sample. That makes percent of tolerance the figure to quote when the gauge sorts parts against a specification. In an expanded study with fewer than ten parts, Minitab strongly recommends %Process against a historical standard deviation over %Study Variation. The ndc is 1.41 times the part standard deviation divided by the GR&R standard deviation, truncated to a whole number. For context on kappa, one published scale describes 0.61 to 0.80 as “substantial” agreement and 0.81 to 1.00 as “almost perfect”. Cohen’s kappa covers two raters and Fleiss’s kappa generalises it to more.
The effectiveness, miss-rate and false-alarm bands are conventions that practitioners attribute to the AIAG manual, and a customer may tighten them. Customer manuals set their own requirements. One supplier manual requires %R&R of 10% or less on critical-to-quality characteristics. Another requires variable studies of at least 10 parts, 2 operators and 3 trials with ndc above 5, and attribute studies of at least 30 pieces, 3 operators and 3 trials with kappa above 0.75. For a vision gauge, read the customer’s specific requirements before you write your own criteria, not after.
What PPAP’s MSA element asks for, and whether the customer approves the protocol first
In the AIAG PPAP 4th edition numbering, Measurement System Analysis Studies is element 8. Element 7 is the Control Plan. One reviewer’s checklist says element 8 needs a study for every gauge used on a Special Characteristic, %GRR within the AIAG range, ndc of 5 or more for variable studies, and a study date that pre-dates the dimensional results. The same checklist says an attribute agreement analysis with effectiveness below 90% on a visual reject gets the package kicked back. The study has to be on the gauge the control plan actually names. A study done on a lab CMM does not cover the caliper the operators use, so the vision gauge needs its own study on the production installation, on-line gauges included. Customer guidelines may also require raw data, work instructions and photos of the gauge and the parts used.
On approval: IATF 16949 allows other methods or criteria if the customer approves them, and the record of that approval is kept with the results. Where a vision gauge needs adapted methods (expanded factors, fixed-factor treatment, combined variable and attribute studies), agree the protocol with the customer before the study runs. Approval obtained afterwards invites the suspicion that the criteria were picked to pass.
Documenting operator and shift reproducibility so it holds up in an audit
Consider the case where the vision system, not the human operator, takes the reading. Reproducibility then has to cover the conditions that do change: who loads the part, which shift, which fixture, which lighting state. The broader definition supports this: reproducibility is variation tied to whatever specified condition changes, including setup and method differences, not just “human error”. Performance qualification for an automated inspection system means showing consistent results across operators, shifts and lots against pre-defined acceptance criteria.
For the record to survive an audit, log instrument ID, fixture, environment, method revision, appraiser, part and trial for every reading, and note anything abnormal before looking at the statistics. Hide part identity and previous values where the method allows, because anchoring makes repeatability look better than it is. Two or three operators give an imprecise estimate of reproducibility, although this matters less when part-to-part variation is much larger. Look at the graphs too: an inconsistent operator can show up there even when the overall numbers pass.
A report structure auditors can follow
A template that one auditor signed off for another company’s system does not guarantee acceptance for a new vision gauge with different defect panels or criteria. What transfers is the structure. Record the intended use, study design, part range, appraisers, analysis method, interpretation basis, known limitations and the conditions under which the conclusion stops applying. A report built on that has these parts:
- Scope: the characteristic, the gauge installation, the control-plan reference and the intended decision.
- Calibration record and Type I bias result.
- Colour robustness audit results and the thresholding decision that followed.
- Study design: factors (colour, wear stage, shift, operator), fixed or random treatment, parts, trials, and the documented rationale for the sample sizes.
- Acceptance criteria with their basis and approval record, defined before the data.
- Raw data, condition logs and anomalies.
- Results for the variable and attribute studies, including the method comparison against the manual gauge.
- Decision, known limitations and re-verification triggers.
The FDA citations Roboflow describes in this area include missing statistical rationale for sample sizes and validation that never checks whether the system reliably rejects out-of-spec product.
When a passed study stops being evidence
A passed gauge R and R describes the system under the conditions it was studied in, and the report should say which changes void it. On a vision gauge, the likeliest are lighting degradation, which can bring back a colour-dependent gap after the first audit passed, fixture wear beyond the interval that was studied, and any change to a per-colour threshold or to the colour-identification logic. Each of these should trigger a defined re-check. The question left for your team is which of those triggers you can detect automatically and which still depend on someone noticing.
FAQ
What does GR&R stand for?
Gauge repeatability and reproducibility: a measurement-system study that estimates how much observed variation comes from repeating the same conditions and how much from changing conditions such as appraisers.
What are PPAP documents?
They are the elements of a Production Part Approval Process submission, from design records and FMEAs through the control plan, MSA studies, dimensional results and the Part Submission Warrant. The submission level sets which elements are submitted and which are retained for the customer to request.
