Machine learning in computer vision fails in four ways that matter more than the choice of model: data that does not represent the cameras and scenes the system will face, labels and test sets that flatter the model, no tolerance for the errors that will happen, and no monitoring once the system is live. Across the ImageNet-C corruption benchmark, Hendrycks and Dietterich found negligible changes in relative corruption robustness going from AlexNet to ResNet classifiers. Real conditions and real scale are what break a demo model, which is the wider argument behind why off-the-shelf CV models fail in production.

Why does a model that worked in development fail on real cameras?

Supervised learning assumes that the training data and the data the model meets in deployment come from the same, static real-world process (the i.i.d. assumption). A 2021 review of machine learning certification calls an unchecked i.i.d. assumption the most frequently encountered example of developers relying on maths that does not hold for their problem. When training and inference distributions differ, the result is a domain gap, and the authors note this is common in real-world applications.

The gap shows up in a few predictable places in vision work.

New locations. Beery, Van Horn and Perona trained detectors on camera-trap images from fixed sites: the algorithms performed excellently at the locations they were trained on, and generalisation to new locations was poor, especially for classification. The WILDS benchmark (Koh et al.) collected ten datasets with shifts that arise naturally, across hospitals, camera traps, and time and place in satellite imagery, and on every one of them standard training gave substantially lower out-of-distribution than in-distribution performance. Existing methods built to handle distribution shift did not close that gap either.

The camera itself. Commonly used public detection datasets contain images that have already been through an image signal processor (ISP) and compression, and ISPs are tuned subjectively by experts for human viewing, often with tens to hundreds of adjustable parameters. A 2023 study in the Journal of Imaging characterised how robust several detection architectures are to changes in those ISP parameters, noting that it is unknown how far an object’s appearance can vary before a learned detector stops finding it.

Changes a human would shrug off. Researchers testing reliability against human perception showed that ResNet50 failed to recognise cars in transformed images that humans had no trouble with. They ran experiments with 2,000 human participants across eight transformations, including brightness, contrast, defocus blur and JPEG compression. Their finding was that standard robustness benchmarks such as ImageNet-C either overshoot or undershoot the range of changes humans tolerate, so the benchmark does not cover the full range of realistic changes that leave a human’s decision unchanged.

The environment. Google Research’s data cascades study (Sambasivan et al., CHI 2021) found that one of the most common causes of cascading data problems is training on noise-free datasets and deploying into a noisy world. In the field, the practitioners interviewed described fingerprints, shadows, dust, poor lighting, and rain and wind physically moving image sensors.

Models learn the site, not the subject

Zech et al. (PLOS Medicine, 2018) trained pneumonia-screening networks on chest radiographs from several hospital systems: a jointly trained model reached an internal AUC of 0.931 and dropped to 0.815 at an external hospital. A network trained just to identify the source hospital did so for 99.95% of NIH radiographs and 99.98% of Mount Sinai radiographs. Because pneumonia prevalence varied widely between sites, the hospital system itself was predictive, and the models exploited it. Within one hospital, scanners from different manufacturers in different wards stored images differently, which the model picked up, and the team only noticed after manual image review. The authors suggest that the networks could rely on subtle differences in acquisition protocol, image processing or compression and overlook the pathology.

If one production line uses a different camera and also happens to have a higher defect rate, a model can learn the camera rather than the defect. Rabanser et al. (NeurIPS 2019) observe that machine learning systems, which depend heavily on properties of their inputs, tend to fail silently, and softmax outputs can extrapolate with unjustified high confidence to data far from the training set.

Before deploying an off-the-shelf or vendor model, test it under realistic conditions on entirely new data never used during development, the condition the certification review sets for its field test.

Which data and labelling mistakes undermine a vision model before model choice matters?

The first check in any vision engagement is the quality and quantity of data, whether the processes that generate more can be reached, and what the system is expected to achieve; that comes before any discussion of models. The measured evidence shows why the data comes first.

Test images that are also training images. Barz and Denzler found that 3.3% and 10% of the test images in CIFAR-10 and CIFAR-100 have near-duplicates in the training set, which rewards memorisation. On a duplicate-free test set, classification accuracy dropped by between 9% and 14% relative to the original figures. In one preterm-delivery study, oversampling rare cases gave near-perfect scores because duplicates sat in both training and test splits.

Wrong labels in the test set. Northcutt et al. estimated an average of at least 3.3% label errors across the test sets of ten widely used computer vision, language and audio datasets, with errors making up at least 6% of the ImageNet validation set. The consequence for model selection is direct: on ImageNet with corrected labels, ResNet-18 outperforms ResNet-50 if the share of originally mislabelled test examples rises by just 6%. A noisy test set can pick the wrong model.

Labels that encode the labelling process. Beyer et al. re-annotated the ImageNet validation set and found recent models’ gains to be substantially smaller on the new labels than on the originals. They concluded that a significant fraction of ImageNet progress came from exploiting biases in the labelling procedure.

Good averages hiding bad subsets. Oakden-Rayner et al. found that hidden stratification (poor performance on subsets nobody identified during training or testing) can produce relative performance differences of over 20% on clinically important subsets. It can occur in subsets with low prevalence, poor label quality, subtle distinguishing features or spurious correlates. On a production line, the rare but costly defect type is the subset to worry about.

Labels made by people outside the domain. The data cascades study found that cascades were set off when practitioners relied on technical expertise rather than engaging with domain experts while discarding, correcting or merging data.

In an industrial flame segmentation study, the researchers suspected the original labels were partly auto-generated and hand-relabelled 200 images to get a usable dataset.

Data checks to run before training anything

Run these checks before training anything:

• Deduplicate across splits so that no image sits in both training and test.
• Confirm no label information leaks into the inputs, such as overlays, file names or burnt-in annotations; label leakage gives high test scores and a model that is useless at inference.
• Audit test-set labels before using test accuracy to choose a model.
• Check that the data represents the task and look for biases and inconsistencies, including in pre-training data.
• Break results down by site, device, lighting and product type, and report the worst slice alongside the average.
• Check whether a trivial feature (camera ID, timestamp, station) predicts the label on its own.

When is classical image processing a better choice than a learned model?

For the highest accuracy deep learning is the leading approach, while traditional image processing remains viable for fast, simple solutions and is more transparent. The comparison has real numbers behind it. In the burner-flame study, segmentation quality rose from 80.3% IoU with global thresholding to 87.0% with a random forest and 93.2% with DeepLabV3+, while CPU time per image rose from 0.1 ms with thresholding to 436.9 ms with the slowest U-Net. Thresholding failed systematically because it only sees grey values and cannot separate bright chamber walls from darker parts of the flame. That is the general weakness of handcrafted features: they struggle with complex visual data, which is the problem CNNs solve by learning hierarchical features from data.

The opposite case is just as concrete. A 2021 comparison of classic and deep methods found that the deep model’s performance collapsed on images outside its training space, while the classic approach stayed robust across imaging modalities. A multispectral registration network trained only on visible-to-near-infrared pairs failed completely when applied elsewhere, whereas the handcrafted descriptor kept working on visible-to-mid-wave-infrared alignment. The same authors note that classic algorithms can come with proven limits, for instance the faintest edge a detector can find given the noise level, which deep models do not offer. For NASA’s Europa Clipper instruments, where compute is scarce, the team used simple threshold or PCA-based techniques for all three anomaly-detection tasks, chosen for robust performance and low compute demand.

On small datasets the sources disagree. A certification review states that random forests and SVMs are superior when no assumptions can be made and only a moderate number of data points are available. The flame study found the opposite on its task: the deep models lost the least performance when trained on less data, helped by ImageNet pre-training. Our reading is that the flame result is one segmentation task with strong pre-trained backbones, so little data alone does not settle the choice; what settles it is whether the target looks like anything a pre-trained network has seen and whether you can afford to label more.

Decision table: classical, learned or hybrid

Condition in the vision system	Leans toward	Basis
Fixed geometry, controlled lighting, clear contrast, CPU-only or sub-millisecond budget	Classical image processing	Thresholding ran at 0.1 ms per image versus hundreds of ms for deep models on CPU
Complex visual data with a representative labelled set	Learned model	Deep models led on accuracy when the training set covers the domain
Imaging modality, sensor or spectrum likely to change after deployment	Classical, or learned with retraining budgeted	Deep models collapsed outside their training space; handcrafted methods held up
Audit or safety case needs known limits and explainable errors	Classical, or classical checks around a learned core	Classic algorithms are more transparent and can have proven detection limits
GPU unavailable at the edge	Classical, or a small learned model	Deep models were slow on CPU in the 2021 comparison
Classical stage localises well but classification is hard	Hybrid pipeline	Hybrid methods have improved performance and handled problems unsuited to deep learning alone

In manufacturing inspection this is the familiar split between rule-based machine vision, which is deterministic and auditable but brittle to variation, and learned computer vision, which adapts but has to be validated in production before it earns trust. The fuller method comparison is in deep learning versus traditional computer vision methods, and a review guide states it plainly: a supervised learning approach should only be chosen once the task cannot be solved well enough without one.

How do you detect drift in a deployed vision system without labels?

Production data distributions are rarely static, and prediction quality can degrade gradually as live data drifts away from the training distribution. The data cascades study found that drift caused by real-world noise takes the longest of the data cascades it studied to surface, up to two to three years, and almost always in production. Yet Rabanser et al. note that ML pipelines rarely inspect incoming data for signs of shift.

A vision model does not crash or write an error log when it degrades: predictions that used to be reliable become wrong slightly more often, in ways too small to see in any single frame. Seasonal light, dust on a lens, a supplier changing a material finish or a worn tool producing a slightly different part profile all move the inputs while the model keeps scoring confidently. On an automated visual inspection line, false positives tend to climb first and operators start ignoring alerts, while missed defects pass quietly until a customer complaint or a rework spike exposes them.

Two kinds of change need telling apart. Covariate shift changes the inputs while the task stays the same (camera angle, image quality, backgrounds, seasonal light, compression, hardware wear); concept or label shift changes what an image means, such as new packaging or revised operating conditions. The fix differs: data drift is usually resolved by retraining on examples from the new conditions, while concept drift, such as a specification change redefining what counts as a defect, needs the ground-truth labels updated first.

Label-free detection works at three levels:

• Input statistics. Compare live distributions of image statistics or embeddings against the training baseline with tests such as the Population Stability Index and Kolmogorov-Smirnov.
• Model outputs. Rabanser et al. found that two-sample tests on the softmax outputs of a pre-trained classifier (black-box shift detection) performed best across the shifts they studied, and that an existing classifier can be turned into a shift detector after the fact. Large shifts were detectable with better than chance accuracy from only 20 samples, while small shifts needed orders of magnitude more.
• Attributions. A change in feature attribution can give early warning of accuracy loss without collecting ground truth, as described for Amazon SageMaker Model Monitor.

Detection is not the same as harm. Distributions shift constantly and often harmlessly, so the useful question is whether a shift is malignant. Rabanser et al. found that two to three orders of magnitude fewer labelled examples than the total sample were enough to approximate accuracy on the shifted data. In practice, that means a small stream of confirmed labels from quality checks the line already runs, used to confirm that a statistical alarm is actually an accuracy problem. Slice the monitoring by site, device, lighting and product type rather than tracking one aggregate number. What this monitoring needs in a factory, from baselines to sign-offs, is set out in the artefacts that keep a line-side inspection model running.

Retrain, roll back or send to review?

Decide by consequence, not by how far the metric moved. A sensible operating pattern is to classify drift by business impact: observe and schedule retraining for low-impact outputs, and respond faster and more conservatively where safety, access, compliance or automated escalation depend on the output. The checks, in order:

1. Confirm it is real. Check whether the drift signal correlates with a drop on a labelled evaluation slice before acting, so noise does not trigger retraining.
2. Rule out the pipeline. Operational changes, such as a format change or a value switching units, are common in industry and can derail a model; reverting the deployment may be the quickest fix.
3. Classify the drift as temporary or structural. A short-lived condition may need only observation, while a new camera fleet, a redesigned product or a permanent workflow change usually needs more than a quick retrain.
4. Escalate high-consequence drift. If drift changes how far a high-consequence output can be trusted, move to human review or roll back before the model keeps influencing decisions.
5. Retrain with a gate. Retrain on fresh, representative data and require the new model to beat the current one on a held-out set before it replaces it. In certified settings, any update triggers a follow-up recertification, and the system is tested and recertified periodically during operation.

Retraining has its own risk: regular retraining can create feedback loops in which the model’s outputs shape its future inputs, intentionally or not.

Designing for errors: fault tolerance and human review

Near-100% accuracy is almost never a plausible target, so the workflow has to tolerate errors and the organisation has to accept a human in the loop. Google’s study of a diabetic retinopathy system deployed in eleven clinics in Thailand found tensions between the model’s thresholds for image quality and the quality of images an imperfect, resource-constrained environment actually produced. A model that rejects real-world inputs as ungradable is a workflow failure even if its accuracy on gradable images is excellent.

Fault tolerance has an engineering shape. In case of failure, failover or switchover paths are essential for hardware and software, and real-time applications should use a monitoring system. For human review, the confidence threshold needs balancing: set too low, the system rarely escalates and misses edge cases; set too high, operators drown in alerts, and finding the balance needs iterative human evaluation. Alarm fatigue can erode the safety the review layer was meant to provide. There is also the automation paradox: the more reliable a system appears, the less prepared the human may be when a rare case needs them. Because softmax confidence can be high on unfamiliar inputs, we would pair confidence-based routing with the drift signals above rather than trust confidence alone.

Checkable standards and test rubrics for vision systems

Standards and rubrics turn “it seems robust” into specific tests. IEEE 3129-2023 provides test specifications with indicators for common corruptions and adversarial attacks for evaluating the robustness of AI-based image recognition services. IEEE P2975.2 defines verification and validation requirements for deep learning models used in manufacturing. Google’s ML Test Score (Breck et al.) sets out 28 specific tests and monitoring needs drawn from production ML systems as a roadmap to production readiness. The human-tolerated range method gives a vision-specific requirement: within the range of image changes that do not alter a human’s decision, the model’s correctness and its predictions should not change either. A 2025 review of industrial computer vision AI standards argues that the build-then-test approach neglects risks after deployment, and calls for lifecycle monitoring and human oversight.

Who can harden a computer vision model for production?

TechnoLynx Computer Vision R&D hardens vision models that worked in development for real cameras and sites, with evaluation, drift and slice-metric monitoring around them. The first thing to check in any vision engagement is the data and the expectations, not the model: what data exists, how more can be generated, and what the business expects.

FAQ

What does computer vision mean in AI?

In its modern form, computer vision is a learned, generalisable software discipline built on convolutional networks and, increasingly, vision transformers. Most production vision models are trained with supervised learning, one of the types of machine learning.