The software does not decide whether an R&R gage study is valid. Two things decide it: a properly designed study, and the right analysis method. For variable measurements, the ANOVA method fits a two-way random-effects model with an operator-by-part interaction term. The result is judged against the AIAG Measurement Systems Analysis (MSA) bands for %GRR and the minimum of 5 distinct categories (ndc). Pass-fail data is different, because each check has only two possible outcomes. There the study becomes an attribute gage R&R study, read through effectiveness, miss rate, false alarm rate and kappa.
A spreadsheet, an R package and a commercial MSA suite will agree with each other if they run the same model with the same settings. None of them will rescue a study with ten near-identical parts or a reference standard nobody checked.
Can I run a Gage R&R in Excel or an open-source package instead of Minitab?
Yes. A spreadsheet, an open-source statistics package and dedicated MSA software all compute the ANOVA variance components behind %GRR and ndc. Real Statistics Using Excel walks through the full analysis as a two-factor ANOVA inside Excel. The SixSigma package for R (ss.rr) returns the ANOVA tables, the variance composition and %Study Var. GageRnR, an MIT-licensed Python library, generates an HTML report from the command line. Templates exist too. The Lean Suite study sheet, for example, uses the AIAG average-and-range method with 10 parts, 2 or 3 appraisers and 2 or 3 trials. Average-and-range is a different method from ANOVA, though, and it cannot detect operator-part interaction.
Saying any tool is equivalent only holds once you look at the settings, because the defaults differ:
- Error term for the F-tests. The usual approach uses the operator-part interaction as the error term, as in appendix A of the AIAG MSA manual and in Minitab. The worked example on page 127 of the same manual uses repeatability instead, which is why the R package added an
errorTermargument in version 0.9-3. - When the interaction is pooled. Minitab drops the interaction from the model when its p-value is 0.05 or higher. The R package uses an
alphaLimargument and, above that limit, pools the interaction into repeatability. The Excel walkthrough makes the same merge by hand when the interaction is not significant. - Study variation multiplier. Minitab multiplies the standard deviation by 6 by default. The AIAG manual notes that 5.15 was the historical convention for a 99% spread. %Tolerance divides study variation by the tolerance, so the multiplier changes %Tolerance.
- Crossed versus nested. The R documentation warns that choosing the wrong design type makes the results meaningless. Nested designs are for destructive tests, where each part can only be measured once.
Spreadsheets deserve one extra caveat. McCullough and Wilson’s audit of Excel 2003 found it better than earlier versions, but not enough to recommend it for statistical work. Their earlier audit of Excel 97 had concluded that Excel should not be used for statistical analysis at all. Before trusting any template, run it on a published worked example, such as Minitab’s crossed study, and confirm it reproduces the published variance components. The Python library also tests the data for normality with a Shapiro-Wilk test, because its documentation says Gauge R&R needs normally distributed data.
Why the interaction term earns the ANOVA method its place
With average-and-range, an operator who measures some parts well and others badly has that pattern folded into appraiser variation (AV). This inflates apparent reproducibility, and a part-feature problem ends up looking like an operator problem. ANOVA separates the interaction, so the significant operator-by-part combination can be targeted directly. The EngiCalcHub guide reports that for simple features such as diameter or length, the two methods usually give %GRR within 1 to 2 percent of each other. It also says ANOVA is strongly preferred for vision systems with feature-finding algorithms, where interaction is more likely.
Reading %GRR and ndc against the AIAG MSA thresholds
No one can tell you whether your numbers pass without first seeing your computed %GRR and ndc and the acceptance criteria that apply to you. Once you have them, the AIAG guidance gives acceptance bands. The table below combines the variable-data and attribute-data criteria.
| Metric | Acceptable | Conditional or marginal | Unacceptable |
|---|---|---|---|
| %GRR (vs total variation or tolerance) | under 10% | 10% to 30% | over 30% |
| ndc | 5 or more | (no band) | under 5 |
| Attribute effectiveness (per appraiser) | over 90% | 80% to 90% | under 80% |
| Miss rate | under 2% | 2% to 5% | over 5% |
| False alarm rate | under 5% | 5% to 10% | over 10% |
| Kappa vs reference | over 0.75, good to excellent | (no band) | under 0.40, poor |
The %GRR bands come from the AIAG MSA manual, 4th edition (2010), and apply both against total variation and against tolerance. The conditional band depends on the application. EngiCalcHub notes that 10 to 30 percent is not acceptable for safety-critical features such as brake dimensions, implants or aerospace fasteners. Some aerospace and medical industries require a 10 percent maximum for safety-critical features. The effectiveness, miss rate and false alarm criteria are from the 3rd edition of the AIAG manual; the kappa cut-offs of 0.75 and 0.40 are the manual’s rule of thumb as quoted by SPC for Excel. Kappa bands vary by source: the CPQP MSA guideline rates 0.60 to 0.80 as good.
For an inspection station that sorts good parts from bad, %GRR against tolerance is the ratio to use. The Lean Suite guidance calculates it as 100 × GRR ÷ (tolerance ÷ 6) and asks you to report which denominator you used. Before reading the total, check its components. A large repeatability error points at the gauge. A large reproducibility error points at operator method or training.
These bands are conventions, not laws. A Quality Digest column argues that the original AIAG procedure has fundamental problems that were never corrected.
How much ndc is acceptable, and what a low ndc means
AIAG recommends 5 or more distinct categories for an acceptable measurement system. The number of distinct categories (ndc) is how many groups the measurement system can tell apart across the range of product variation. Minitab calculates it as 1.41 times the part standard deviation divided by the gauge standard deviation, truncated, with a floor of 1. At 2, it can split parts only into high and low.
Minitab’s own crossed example shows how a study can fail on ndc while its %GRR looks borderline. Total Gage R&R is 27.86% of study variation, which might be acceptable depending on the application, yet the ndc of 4 falls short of the AIAG minimum of 5.
A low ndc is also a check on part selection, not only on the gauge. Real Statistics treats ndc as a check on whether the parts chosen have enough variability, and the CPQP guideline says a study with ndc below 5 should start over with more parts spanning the full specification limits. Someone running a pass-fail check may argue that two categories are all it needs. The flaw is that ndc’s two groups are high and low across the range of product variation. A sorting gauge is judged on %GRR against tolerance, and a go/no-go gauge by an attribute study.
How many parts, operators and trials an R&R gage study needs
The standard variable study uses 10 parts, 3 operators and 3 trials, which makes 90 measurements. The parts are chosen to span the tolerance, operators work blind, and the order is randomised. The parts should cover 80 to 100 percent of the tolerance, even if that means using scrap from both ends. The CPQP guideline gives 10 parts as the minimum with three operators taking three measurements each. The NIST/SEMATECH e-Handbook adds that only trained, experienced operators should take part. If only a few people use the gauge, all of them should be included. If the study must represent a large pool, a random sample of more than two should be drawn.
Ten parts is a convention, not a precision guarantee. Minitab’s Assistant white paper found that 10 parts, 3 operators and 2 replicates leave a 35 to 40 percent margin of error on the part-to-part standard deviation at 90 percent confidence. About 35 parts are needed for 20 percent precision at 90 percent confidence, and about 135 for 10 percent. Under typical settings, the repeatability estimate is reasonably precise. Its degrees of freedom are parts × operators × (replicates − 1), so 10 × 3 × 1 = 30, close to the roughly 35 needed for a 20 percent margin. Two or three operators do not estimate reproducibility precisely, although this matters less when part variation is much larger than operator variation.
This imprecision affects the decision about whether the gauge is acceptable. If a 10-part study lands near 30%, the honest next step is more parts, not a second run on the same 10. EngiCalcHub puts it bluntly: the hard part of Gage R&R is study design (part selection, operators, randomisation, blind labelling), not the arithmetic.
How is a Gage R&R done for pass-fail inspection?
Pass-fail inspection produces attribute data: each check has only two possible outcomes, such as go/no-go. Attribute data calls for an attribute study instead of the ANOVA method. The AIAG MSA 4th edition quick guide lists signal detection and hypothesis test analyses for basic attribute systems, alongside range, average-and-range and ANOVA for variable systems.
Minitab’s guidance asks for:
- at least 50 samples
- at least 3 appraisers, ideally 3 to 5, chosen to represent everyone who uses the system
- each sample rated at least twice, in random order within each replicate
- a balanced design, where every appraiser rates every sample the same number of times
- several samples that are marginally acceptable and several marginally unacceptable
A panel of obvious passes and obvious gross defects will flatter any inspector, human or machine.
Agreement is also scored against a known reference. In the SPC for Excel worked example, 30 parts were first classified with a variable gage (21 conforming, 9 nonconforming), and three appraisers then rated each part three times with a go/no-go gauge. Kappa can then measure each appraiser’s consistency with themselves, agreement between appraisers, and agreement with the reference. When the standard is known, Minitab calculates kappa for each trial against the standard and takes the mean.
Worked example: one appraiser’s miss rate against the AIAG table
Miss rate is the percentage of times an appraiser fails to reject a nonconforming part. In the SPC for Excel study, 9 nonconforming parts × 3 trials gave each appraiser 27 opportunities to miss. Bob missed twice, so his miss rate is 100 × 2 ÷ 27 = 7.4%. Against the AIAG criteria, anything above 5% is unacceptable for the appraiser. Only 22 of the 30 parts got the same rating from every appraiser on every trial, an overall agreement of 73.3%, with a 95% confidence interval from 54.1% to 87.7%. That interval is 33.6 points wide.
Comparing a vision inspection system against certified human inspectors
Score the vision system and the human inspectors against the same verified reference. Do not treat the humans as the reference. Human visual inspection has been measured well short of ground truth. A 2022 study in the MDPI journal Safety applied attribute agreement analysis to aircraft engine blade inspection. Operators were on average 82.5% consistent with their own decisions, achieved 67.7% inspection accuracy, and 20 operators (40%) met the required standard. Comparing a machine with a panel that variable would measure the panel’s disagreement as much as the machine’s error.
In practice, treat the vision system as one more appraiser in the attribute study. If the system produces a continuous measurement or score before it thresholds to pass or fail, run a variable ANOVA study on that value as well. In that study, fixtures or stations play the role operators play in a manual study. The NIST guidance on sampling gauges follows the same logic: include every gauge if there are few, and sample at random if there are many. This is where EngiCalcHub’s preference for ANOVA on vision systems applies.
Two neighbouring problems need their own treatment. One is reconciling the certified inspectors’ calls into a ground truth that separates inspector disagreement from model error. The other is a pre-registered equivalence or non-inferiority protocol for the human-versus-machine comparison. When a vision system replaces calliper or micrometer measurement, see our guide to designing a defensible Gauge R&R study for vision inspection. For the wider industry context, see our page on AI engineering for manufacturing and automotive.
What does an auditor expect to see in a Gage R&R report?
A report that holds up whatever tool produced it carries:
- %GRR, %EV, %AV, %PV and ndc, with a bar chart of the variance components
- the ANOVA table, which most automotive OEMs require for PPAP submissions, and the operator-by-part interaction p-value
- a statement of whether %GRR is computed against total variation or against tolerance
- a clear statement of any approach that differs from the AIAG manual, which the manual itself requires in results given to the customer
- the study design: part count and how the parts span the tolerance, operators, trials, randomisation and blind labelling
- for pass-fail stations, effectiveness, miss rate and false alarm rate per appraiser, plus kappa against the reference
An auditor who can see that the ANOVA settings match the manual and that the template was checked against a known result has no reason to care which product drew the charts.
FAQ
Is it gage R&R or gauge R&R?
Both spellings refer to the same study. “Gage” is the spelling used in the AIAG manual and US automotive practice, and “gauge” is the usual British spelling.
What are the three elements to be checked in a gage R&R?
Repeatability (equipment variation), reproducibility (appraiser variation) and part-to-part variation. The first two together make up GRR, which is compared against part variation and total variation.
