You can trust vision inspection equipment once a documented characterisation protocol has been run on your own line and you can repeat it. A vendor’s lab sheet does not give you that. EMVA 1288 defines how to measure and present sensor and camera parameters for machine vision. ISO 12233 defines how to measure resolution and spatial frequency response. Uniform illumination underpins a large share of characterisation and calibration work. On top of those measurements sit three more layers: a staged pilot with your own parts, an acceptance test that controls the final payment, and a written trace from every optics and lighting choice back to the smallest defect the system must catch.

The characterisation protocol for vision inspection equipment, at a glance

The table below gives the order we would run the steps in, the reference method for each, and the evidence each step should leave behind.

Step	Reference method	What it controls	Evidence to keep
Sensor parameters	EMVA 1288: homogeneous source at f/8, illumination non-uniformity of 3% or less	Sensitivity, noise, dynamic range and non-uniformity, measured under identical stimulus	One datasheet per documented operating point (gain, offset, exposure, mode)
Resolution and depth of field	ISO 12233 slanted-edge e-SFR, with 4:1 edge contrast in the 2014 revision	Sharpness and depth of field across working distances	MTF against distance for each camera
Illumination	Flat-field capture, then capture at line rate with the production trigger	Vignetting, shading, pulse-to-pulse variation	Frame-to-frame and across-field uniformity records
Temperature	Chamber mapped at the test setpoints, both empty and loaded	Chamber gradients, plus drift in the sensor, optics and fixtures	Mapping report and detection rate at each setpoint
Traceability	Requirement specification, then system specification, then test matrix	Every optics choice justified by the defect requirement	Matrix linking each requirement to a test, sample, result and record
Pilot	Blind challenge set, with the confidence method defined before testing	Performance on your own parts, on the live line	Reconciled decisions, rejects and recovered samples
Acceptance	IQ/OQ/PQ adapted into a site acceptance test, VDI/VDE/VDMA 2632-3 for classifying systems	Final payment depends on on-site performance	Validation summary report

Checking MTF and depth of field on every camera, from smallest part to largest

A depth-of-field figure only means something when it comes with the resolution and contrast it guarantees. For telecentric lenses, Vision & Control gives a typical working definition: the MTF should still be about 20% at the edges of the depth-of-field range. The same source points out that depth of field and resolution trade against each other, because their product stays constant. A station that inspects a wide range of part sizes therefore trades resolution against depth of field.

The measurement is a batch of slanted-edge images taken at a series of distances, analysed for sharpness at each position. Imatest’s FocusField postprocessor is built to do exactly that. Imatest recommends MTF50P as its sharpness metric. The slanted-edge method itself needs care. Non-uniform illumination can push the computed e-SFR up or down, depending on which side of the edge the light falls off. The edge angle introduces a systematic error as it moves away from the sampling grid towards 45 degrees. Imatest also found that a setup meeting the ISO measurement conditions still showed significant variability in certain data sets from the variables left uncontrolled. Lighting, chart angle, processing settings and distance therefore all go into the test record.

On a four-camera telecentric station with a strobed ring light inspecting M3 to M8 fasteners, run the slanted-edge check on each camera separately, with the target at the focal positions of both the M3 and the M8 part, and set the pass criterion as a stated MTF at both extremes rather than a nominal depth of field from the lens datasheet. Measuring every camera matters because a chart sized for one end of the range may not stress the other end.

How do you prove strobed lighting stays uniform at line rate, not just on the bench?

Line-rate uniformity needs a dynamic measurement, because a static flat field cannot show timing jitter or pulse-to-pulse variation.

At 40 parts per second we would verify flatness with the production trigger chain at line rate, capturing enough frames to separate across-field from frame-to-frame variation. TechnoLynx can define what flatness means for a given defect contrast, design and implement the at-speed procedure on the target cameras, lighting and triggering, and confirm the tolerance against known defect samples. The capture equipment used for the dynamic measurement needs validating too. Set the flatness tolerance for thread-defect contrast from known defect samples on your own line.

A static flat-field measurement shows vignetting and non-uniformity, whether they come from the lens, the illumination or the sensor. As a static capture, it cannot show what happens when the strobe fires in time with moving parts. In TDI imaging, a strobe controller has to reproduce every pulse with consistent timing and intensity, and pulse-to-pulse timing jitter changes the phase between illumination and acquisition. There, even small timing errors produce brightness non-uniformity.

The stakes are higher when a trained model sits downstream. In line-scan imaging, a brightness gradient of two or three percent appears in the image as a systematic artifact that looks like a real surface feature. A neural network can learn illumination gradients and gain variation as features of its training data. When the production imaging drifts away from those conditions, detection performance degrades in ways that are hard to diagnose.

Setting up a bench rig that does not bias the sensor comparison

To remove rig bias, build one controlled bench and apply one calibration procedure identically to every sensor sample before any comparison measurement. Do not recalibrate each sample with different settings, and do not compare raw outputs from an uncalibrated rig. Both reintroduce the bias you are trying to remove.

For the sensor itself, EMVA 1288 sets the reference geometry. The bare sensor, with no lens, is illuminated by a diffuse disk-shaped source, and the measurements require an f-number of 8. The standard recommends illumination non-uniformity of no more than 3%. Irradiance is calibrated absolutely with a calibrated photodiode placed where the sensor sits. At least 50 roughly equally spaced exposure levels are used, and production measurements may drop to 9. Non-uniformity values come from images averaged over many frames, typically 100 to 500, because temporal noise is otherwise as large as or larger than the spatial variation being measured.

Bias also hides in the light source. DXOMARK points out that lens shading correction, optical centre calibration and defective pixel screening all assume even light across the field. When the light is uneven, the correction tables record the flaws of the light source instead of the behaviour of the camera. Once lenses are mounted, extend the same discipline to a chart-based calibration run against one fixed reference chart. The chart’s quality, its illumination and its position then become variables to control, or they become a new confound.

Calibration removes systematic rig bias. It does not remove the sensor-to-sensor manufacturing variation that the comparison exists to measure. NIST’s guidance on multi-sensor work makes the same point: data from several sensors need the same traceability, and the differences between instruments need to be understood.

How many production samples rule out a cherry-picked evaluation unit?

EMVA 1288 sets no sample count, which is why the question comes up. The standard leaves it to the datasheet publisher to decide whether the data is typical, from one individual component, or guaranteed. A datasheet is EMVA 1288 compliant if the mandatory results from at least one camera are reported. A compliant datasheet can therefore describe a single unit. The useful evidence is the spread across units drawn from separate production lots. Repeated measurements and their standard deviation give a Type A uncertainty estimate. A handful of units tells you whether a spread exists. It does not put a statistical bound on that spread, and a claim about worst-case units needs a sample size chosen with that claim in mind.

Which temperature points a reduced thermal plan should keep

In the EMVA 1288 sensor model, temperature has a narrow effect. EMVA 1288 measures dark current at different ambient temperatures because it is the only camera parameter with a strong temperature dependence. For that measurement, the 4.0 Linear release asks for at least six equally spaced temperatures across the camera’s operating range. Detection rate is a different matter, because it depends on the whole system. In gauging applications, temperature drift can come as much from the part and the fixturing as from the vision hardware, since each expands at a different rate.

The test chamber itself contributes to the measurement uncertainty. One chamber guide describes mapping with nine calibrated sensors at -40 °C, 25 °C and 85 °C, passing at a spread of 2.0 °C or less, with samples taking up no more than two-thirds of the internal volume. IEC 60068-3-5 performs its confirmation measurements with an empty working space. EURAMET guidance notes that loading changes the spatial distribution of temperature, and that a loaded chamber has higher uncertainty than an empty one. Running several sensors in parallel therefore calls for a chamber mapping done with the fixtures in place.

Our judgement, with limited chamber time: keep -40 °C and 85 °C, add any transition zone that earlier characterisation flagged, and run sensors in parallel only if loaded mapping shows the chamber stays uniform with the fixtures in. Intermediate points get whatever chamber time is left.

Tracing optics and lighting specs back to the smallest critical defect

A traceable inspection configuration is one where each resolution, lens and lighting choice can be followed back to the smallest critical defect through documented requirements, and forward to the challenge tests that showed it is detected.

The chain starts from the defect. Resolution should be derived from the smallest feature that must be distinguished across the field of view, with allowances for focus, contrast, motion and processing. Megapixel count alone does not establish capability. Vision & Control suggests at least 20% contrast for a structure to remain resolvable. VDI/VDE/VDMA 2632 Part 2 gives guidance on writing the requirement specification and system specification for industrial image processing systems, with emphasis on influencing factors and their effects. The test end of the chain is a matrix linking each approved requirement to a test, a sample, an expected result and a retained record.

NIST’s guidance on calibrating electro-optical sensors lists what a traceability claim needs. To claim traceability, you need a clearly defined measured quantity, a full description of the measurement system, a result with documented uncertainty, and a specification of the reference at the time of comparison. Traceable records of the defect standards, the measurement procedure and the uncertainty make results robust enough for regulatory or commercial decisions.

EU GMP Annex 1 asks for a defect library covering all known defect classes, and operator qualification that considers worst cases such as line speed. The FDA’s draft guidance on injectables adds that defect libraries should include examples from the lower limits of detection. Challenge sets that only contain typical defect sizes therefore miss these lower-limit cases. Once a camera, lens or light changes after qualification, the matrix needs reassessing. That is the subject of change control across optics, lighting and algorithm, and any such hardware swap calls for a documented impact assessment and partial or full revalidation.

Can we pilot one camera on the live line before buying all six?

Yes, provided the vendor agrees. A single-camera pilot on the live line, run with your own parts and a blind test set, is the sensible go/no-go gate before the remaining cameras are bought. Whether the vendor accepts a blind test is a contract negotiation, not a technical given. If the vendor refuses blind testing, or the pilot sample is too small to support the claim, the business case is still unproven.

The challenge set should represent the product and process window, not just ideal samples. It should include acceptable good variation, each relevant defect class and boundary examples, presented in a randomised or blinded sequence. Afterwards, every presented sample is reconciled against the system’s decisions and the physical rejects. The number of challenges should be justified by risk, expected defect frequency and the confidence required. Repeating one obvious defect many times is not the same as testing several defect modes across the operating range. If a statistical claim is wanted, the confidence method has to be defined before testing starts.

Worked example: where the 299-part figure comes from

The success-run theorem gives a zero-failure sample size: n = ln(1 − C) / ln(R). For 95% confidence (C = 0.95) at 99% reliability (R = 0.99), n = ln(0.05) / ln(0.99) = −2.996 / −0.01005 ≈ 298.1. Rounding up gives 299 parts tested without a single failure. Durivage (2016) states that 298 parts cannot support the 95/99 claim, so 299 are required. The assumption to state openly: applied to detection, this means about 299 presentations of a given defect class with no misses before you can claim a 99% detection probability at 95% confidence for that class. Several defect classes multiply the effort, which is why the defect taxonomy (which defects count, and how they are graded) should be agreed before the pilot is sized. The IPA best-practice document on sterile products assigns an AQL of zero to critical defects and progressively less stringent values to major and minor defects.

A pilot on one camera cannot show interactions or throughput problems that only appear when all six cameras run together. Full-system throughput still needs its own check before final acceptance.

Making final payment depend on an on-site acceptance test

Final payment can depend on an on-site acceptance test if the contract says so. Use a lean IQ/OQ-style qualification protocol as the test, preferably piloted on one or two cameras first, with the accuracy threshold and defect set taken from your own line rather than the vendor’s lab. TechnoLynx can turn the vendor’s accuracy claims and your defect taxonomy into a measurable acceptance specification, set up the pilot rig and any inference pipeline it needs, run the test on site, and carry the protocol forward as the system scales to all six cameras.

The contract mechanism is an established clause type. A site acceptance test clause defines how delivered equipment is tested at the buyer’s site against agreed specifications, and passing it is often a prerequisite for final payment or formal acceptance. The clause needs clear definitions of the pass and fail criteria and the test conditions. Treat this as an engineering structure for the agreement, not legal advice.

For the test content, a structured validation follows the V-model lifecycle. It starts with a user requirements specification whose measurable criteria (minimum detection rate, maximum false-reject rate, cycle time, environmental range) become the pass/fail thresholds. Installation qualification follows, then operational qualification with golden, defect and boundary samples, then performance qualification under production conditions at a statistically significant sample size. Unregulated manufacturing may only need an OQ-equivalent functional acceptance test. For classifying systems, VDI/VDE/VDMA 2632 Part 3 defines characteristics of classification performance and procedures for assessing it during acceptance. Measuring systems already have established capability parameters; classifying systems lacked them until this standard. For any gauging functions, the A3 guidance suggests ten parts covering the tolerance range, each run through the system 30 times, fixtured as in production. A gage capability index under 10% of part tolerance indicates the vision system is not distorting results. A Cpk of at least 1.33 is a common automotive production benchmark under AIAG PPAP.

A pilot on one or two cameras may not represent the variation across all six camera positions and lighting conditions. The defect set also needs updating whenever the product mix changes.

A sensor comparison protocol your team can rerun without the vendor

A fair comparison of OmniVision, onsemi and Sony sensors under low-light and HDR conditions comes from a documented bench protocol. It fixes the illumination levels, charts, exposure, gain and processing settings, and the analysis method, so that every sensor sees the same stimulus. A protocol like that is vendor-neutral and can be repeated in-house. LynxBenchAI applies measurement to the compute side: throughput, latency, memory and energy. It publishes measured results for the AI hardware that would run the inspection model. It does not characterise image sensors, so the sensor bench described here remains a separate piece of work; see the LynxBenchAI hardware results when choosing the inference device.

The standards divide the work between them. ISO 12233 covers resolution and spatial frequency response, ISO 15739 covers noise against signal level and dynamic range, and EMVA 1288 covers machine vision sensor specifications. For low light, the relevant EMVA quantities are the absolute sensitivity threshold (the photon count at which SNR equals one) and the dynamic range. The goal of EMVA 1288 is reproducible procedures that make cameras easier to compare.

Two details decide whether the comparison is fair. The first is processing. EMVA 1288 4.0 Linear cannot be used once preprocessing such as debayering, denoising or edge sharpening has been applied. Release 4.0 General instead treats the camera as a black box with no linearity assumption, and it needs more exposure steps for HDR cameras. Raw and ISP-processed outputs therefore have to be compared like for like. The second detail is documentation. EMVA requires the operating points (gain, offset, exposure, camera mode) to be described fully enough that anyone reading the datasheet can repeat the measurement. That same requirement is what lets your own team rerun the protocol without the vendor present.

Repeatability still has to be shown rather than assumed. Repeat runs across operators and sessions come before any conclusion about which vendor is better. Charts and light levels should also match the final application, or the comparison will not predict field performance.

Do IP ratings settle whether a camera survives condensation?

No. A nominal IP rating does not capture thermal cycling or condensation getting in over time; see testing vision hardware where it actually lives.

Who can help us validate vision inspection equipment before we sign?

TechnoLynx turns vendor claims and your defect taxonomy into a written acceptance protocol and runs it on site, though a team with optics metrology skills, bench time and a stable defect taxonomy can run much of this protocol itself. A specialist earns its place when the protocol has to stand up in a payment dispute or an audit, or when the line cannot spare the engineers. When choosing a partner, look for one that writes the procedure down so you keep it, separates rig bias from sensor variation, and is independent of the camera vendor. See AI engineering for manufacturing and automotive for how that work fits a production line.

The limits are just as clear. Accredited reference laboratories sit in the calibration chain below the national metrology institute. Teams that only need a datasheet check on a commodity camera for a non-critical station are better served by the vendor’s own EMVA 1288 data.

FAQ

How often does a machine vision camera need calibrating?

The A3 guidance suggests calibration may be needed daily, with a full accuracy and repeatability test perhaps monthly if the calibrations show no change. Recalibration should also follow any line change that affects the optical path or the scene.

What is the NIST standard for calibration traceability?

NIST policy treats a measurement as traceable only if it can be related to a national or international standard through an unbroken chain of measurements, each with documented uncertainty.