How an Image Detection Model Works in Industrial Inspection

An image detection model does two jobs at once on every frame: it decides what is in the image, and it decides where. On an inspection line, that second job — drawing a box around the defect, not just labelling the part good or bad — is the part that matters most, and it is also the part that a single headline accuracy number tells you nothing about.

Most teams evaluating computer-vision inspection treat the model as a box that either works or doesn’t. A vendor quotes 98% accuracy, someone writes it into a slide, and the assumption is that the number transfers to their defect set. It almost never does cleanly. The accuracy was measured on curated public data with even lighting and balanced classes; your line has glare off a stamped surface, a fixture that shifts part orientation, and a scratch class that shows up in one part per ten thousand. Understanding what the model actually computes is what lets you read a feasibility audit’s per-defect accuracy bands instead of trusting a marketing figure.

What Does an Image Detection Model Actually Do?

Start with the distinction that a “detection” model draws against its neighbours, because the word “detection” gets used loosely.

A classification model answers one question about the whole image: which class does this belong to? Defective or not; scratched, dented, or clean. It returns a label and a confidence, and nothing about location. A detection model answers a harder question: for each object of interest, what is it and where is its bounding box? A segmentation model goes further still, labelling the image pixel by pixel so the boundary of the defect is traced, not boxed.

For defect inspection, that difference is not academic. If you only classify, you know a part is bad but not where the flaw is — which makes root-cause analysis and process feedback almost impossible. Detection gives you localisation, which is usually the minimum useful output: it tells the SPC system where on the part the defect landed. Segmentation gives you area and shape, which matters when a defect’s severity depends on its size. We walk through where each fits in more detail in our explainer on instance segmentation models for manufacturing inspection; the short version is that you choose the task by what decision the output has to support, not by which is most sophisticated.

Under the hood, a modern single-stage detector like YOLO passes the image through a convolutional backbone that extracts features at several scales, then a detection head that, for each spatial location, predicts a set of candidate boxes with a class score and an objectness score. A two-stage detector such as R-CNN first proposes regions likely to contain an object, then classifies and refines each proposal — slower, historically more precise on small objects. The frameworks behind this are ordinary: PyTorch for training, TensorRT or ONNX Runtime for deployment. Nothing exotic. The engineering difficulty is not the architecture; it is matching the model’s behaviour to a line the benchmark never saw.

How a Detection Model Localises a Defect Versus Just Labelling It

Localisation is the mechanism that separates detection from classification, so it is worth being precise about what the model emits.

For every predicted box, the model produces four coordinates (the box geometry), a class distribution (what kind of defect), and a confidence score that combines how sure it is that something is there with how sure it is about the class. A post-processing step called non-maximum suppression then removes overlapping boxes that describe the same defect, keeping the highest-scoring one. The output you actually consume is a list of boxes, each with a class and a confidence between 0 and 1.

That confidence score is the lever the whole deployment turns on. You set a threshold: keep every detection above it, discard the rest. Set it high and you report only the defects the model is very sure about — few false alarms, but you miss the marginal ones. Set it low and you catch more real defects at the cost of flagging good parts. There is no universally correct threshold; there is only the one that meets your false-positive ceiling at your line speed. Reading the box coordinates and confidence together is exactly the literacy that makes a production-line object detection pipeline legible rather than magical.

What a Benchmark Accuracy Number Actually Measures

Here is the trap. When a model reports “98%,” that figure is almost always a variant of mean average precision (mAP) computed on a public dataset — COCO, or a curated defect corpus — where the classes are balanced and the images are clean. mAP itself is a composite: it measures how well predicted boxes overlap ground-truth boxes (intersection-over-union) and how well the confidence ranking separates true from false detections, averaged across classes. It is a good research metric. It is a poor procurement metric taken alone.

The reason is structural, not a matter of the model being weak. Benchmark accuracy is measured under conditions your production line breaks:

Class balance. Benchmarks have roughly even class counts. Your defect distribution is long-tailed — the dangerous defect is the rare one, and mAP barely moves when the model fails on a class that is 0.1% of the images.
Lighting and surface. Public images are evenly lit. A glare hotspot on a machined face can look like a scratch, or hide one.
Fixturing. Benchmark objects appear in varied but natural poses. Your part sits in a fixture; if the fixture drifts a few millimetres the model sees a distribution it was never trained on.
IoU threshold. A number quoted at mAP@50 (boxes counted correct at 50% overlap) is far more forgiving than mAP@50-95, which averages across stricter overlap requirements.

That last point matters enough that we treat it separately: the gap between what mAP@50 means for defect detection and the stricter averaged metric is where a lot of “the model looked great in the demo” disappointments originate. A model can score 0.95 mAP@50 and 0.6 mAP@50-95 on the same data — the second number tells you it localises loosely, which for a small surface defect can mean the box misses the flaw.

None of these figures are lies. They answer a question — how well does this model rank and localise on this curated set — that is simply not the question your line asks. This is the reasoning class of a benchmark measurement being read as if it were an observed-pattern on your equipment: two different evidence classes, and the substitution is where feasibility estimates go wrong.

Reading Detection Output: A Diagnostic Rubric

Use this to interrogate any detection result — a vendor demo or a feasibility-audit table — before you trust it.

Question to ask	What a strong answer looks like	Red flag
On whose data was accuracy measured?	Your parts, your lighting, held-out set	A public benchmark score only
Which metric, at which IoU?	mAP@50-95 per defect class, plus recall at a fixed false-positive rate	A single blended “accuracy %”
How is the rare defect class scored?	Reported separately, not averaged away	Only the aggregate mAP
What confidence threshold was assumed?	Named, with the resulting FP rate at line speed	Threshold unstated
How was localisation validated?	Boxes checked against ground-truth overlap	“It found the defect” with no IoU
What breaks it?	Named failure modes: glare, fixture drift, occlusion	“It’s robust”

If most answers fall in the middle column, the number is actionable. If they fall in the right column, you have a demo, not an estimate.

How Confidence Thresholds Trade Detection Rate Against False Positives

The threshold decision is where model literacy converts into a business number, so work a concrete example. Suppose a feasibility audit reports, for your scratch class, that at a confidence threshold of 0.5 the model detects 96% of scratches but flags 4% of good parts as defective; at 0.7 it detects 90% but flags only 1%. (Illustrative figures — the shape is what matters.)

At line speed those percentages become throughput and cost. If your false-positive ceiling is 1% — the rate above which manual re-inspection eats the savings — then 0.7 is your operating point, and you accept that the model misses one scratch in ten. Whether that is acceptable depends on the cost of an escaped defect versus a re-inspection. That trade is a business decision the model cannot make for you; it can only give you the curve. The threshold band where CV stays cheaper than manual inspection is the number a feasibility audit exists to produce, and you can only read it if you understand that detection rate and false-positive rate move in opposite directions along the confidence axis.

This is also why “highest benchmark score wins” is the wrong selection rule. Two models can have identical mAP and completely different threshold curves on your defects — one gives you a clean 1%-FP operating point, the other doesn’t have one at all. We see this pattern regularly when a team ports a model that topped a public leaderboard onto real fixtures.

Why Lighting, Fixturing, and Long-Tail Defects Break Good Models

A model that scored well on public data has learned the statistics of that data. When your line presents a distribution it never saw — a reflective surface, a shadow from a new fixture, a defect morphology that appears once a shift — the model extrapolates, and detection models extrapolate badly on the tail. This is not a bug to be patched; it is the reason data collection on your actual line is the expensive, unavoidable part of a deployment.

Two responses help. Fine-tuning on line-representative images teaches the model your surfaces and your rare classes — we cover the scope and limits of that in fine-tuning YOLO for manufacturing-line defect detection. And controlling the imaging — fixed lighting, consistent fixturing — narrows the distribution the model has to handle in the first place, often doing more for reliability than a bigger network. Deciding which model architecture and imaging setup fit a given line is the kind of question our computer vision practice works through as a feasibility assessment, and it connects directly to the porting-and-performance methodology we apply across our engineering services.

FAQ

What should you know about an image detection model in practice?

A detection model runs each image through a feature-extracting backbone and a detection head that, for every region, predicts a bounding box, a defect class, and a confidence score. In practice it outputs a list of boxes with classes and confidences; you set a threshold that keeps the confident detections and discards the rest, which is the single decision that governs how the model behaves on your line.

What is the difference between image classification, detection, and segmentation for defect inspection?

Classification labels the whole image (part good or bad) with no location. Detection adds localisation — it draws a bounding box around each defect and labels it. Segmentation labels the image pixel by pixel, tracing the defect’s exact boundary and area. You choose the task by what decision the output must support: root-cause needs location (detection), severity-by-size needs area (segmentation).

How does a detection model localise a defect versus just labelling the image?

For each candidate, the model predicts four box coordinates alongside the class and confidence, then non-maximum suppression removes duplicate boxes describing the same defect. The result tells you where on the part the flaw sits, not merely that the part is defective — which is what lets an SPC system act on the finding.

What does a benchmark accuracy number actually measure, and why can it mislead on a production line?

A headline figure is usually mean average precision on curated public data with balanced classes and clean lighting. It measures box overlap and confidence ranking on that set — not on your parts. It misleads because your line has long-tailed defects, glare, and fixture drift that the benchmark never presented, and a single blended percentage hides poor performance on the rare, dangerous class.

How do confidence thresholds trade off detection rate against false positives for industrial defects?

Raising the threshold keeps only high-confidence detections: fewer false alarms but more missed defects. Lowering it catches more real defects but flags more good parts. The two rates move in opposite directions along the confidence axis, so the correct threshold is the one that meets your false-positive ceiling at line speed — a business trade the model can quantify but not decide.

Why do lighting, fixturing, and long-tail defects break models that scored well on public benchmarks?

A benchmark-strong model has learned the statistics of clean, balanced public data and extrapolates poorly to distributions it never saw. Reflective surfaces, shadows from a shifted fixture, and defects that appear once a shift are exactly that unseen tail. Fine-tuning on line-representative images and controlling the imaging both narrow the gap.

How do I read the per-defect-class accuracy bands from a feasibility audit?

Read each class separately, never the aggregate. For each defect class, look for the metric and IoU used (prefer mAP@50-95 and recall at a fixed false-positive rate), the assumed confidence threshold with its resulting false-positive rate at line speed, and the named failure modes. Those bands tell you which defects are production-ready and which are still a research problem.

The useful question is not “how accurate is the model” but “at what confidence threshold does this model hold my false-positive ceiling on my worst defect class, at my line speed?” A model that cannot answer that has not been evaluated on your problem — and reading a feasibility audit’s per-defect bands, rather than a headline number, is how you get the answer before committing the line.