Production image processing is not benchmark image processing The gap between research benchmarks and production performance is wider in image processing than in most machine learning domains. ImageNet top-1 accuracy tells you how a model performs on a well-curated, well-balanced, well-labelled dataset. It tells you very little about how it performs on your specific imaging hardware, under your lighting conditions, on your subject population, after six months of production operation. This article covers the practical engineering decisions for deep learning image processing systems that need to run reliably in production: model architecture selection, training data requirements, augmentation strategy, deployment optimisation, and managing the distribution shift that happens over time. The architecture decisions discussed here sit inside the model-inference stage of a larger CV system; the broader question of how that stage interacts with ingestion, preprocessing, and post-processing is treated in our methodology for architecting modular computer vision pipelines for production reliability. CNN vs Vision Transformer: which architecture wins in production? The two dominant architecture families for image processing tasks are Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs). The choice between them is not obvious. It depends on training data availability, latency requirements, and task structure β not on which architecture is fashionable in the current quarterβs papers. Property CNN Vision Transformer (ViT) Inductive biases Strong (locality, translation equivariance) Weak β relies on data to learn structure Training data requirement Lower β inductive biases help with less data Higher β needs large datasets to learn spatial relationships Performance at scale Saturates earlier with data scale Continues to improve with more data Inference latency Lower β highly optimised CUDA kernels Higher β attention is compute-intensive Hardware efficiency Excellent on GPU and CPU Excellent on GPU; less efficient on CPU and embedded hardware Transfer learning Excellent Excellent when pretrained at scale (DINOv2, SAM) Interpretability Moderate (CAM, GradCAM) Moderate (attention maps) Small image size Handles well ViT patch size must be tuned; poor on very small images In our experience, CNNs remain the practical default for production image processing where: Training data is limited (under ~100k labelled samples). Inference must run on CPU or embedded hardware. Latency is a hard constraint (under 20 ms per image). The task is well-defined classification or detection. ViTs are worth evaluating when large-scale pretraining is available for the domain (medical imaging, satellite imagery), training data is abundant, GPU inference is acceptable, and the task requires global context understanding β for instance, anomaly detection that depends on relationships across the full image rather than local texture. Hybrid architectures (EfficientNet, ConvNeXt, MobileNetV3) offer competitive performance with deployment-friendly characteristics and are often the best practical choice when neither a pure CNN nor a ViT clearly fits the requirements. The decision is rarely made on benchmark accuracy alone. It is made by combining the latency budget, the available labelled data, and the inference hardware, then choosing the simplest architecture that satisfies all three. How much training data do you actually need? Data requirements scale with task complexity and the degree of visual variation in the deployment environment. The numbers below are observed-pattern figures from production engagements rather than benchmark results β they assume a pretrained backbone and competent augmentation, and they will shift for any project with unusual appearance variation. Task Minimum training samples Notes Binary classification (two well-separated classes) 500β2,000 per class With pretrained backbone; more for complex appearance variation Multi-class classification (5β20 classes) 1,000β5,000 per class More classes β more samples for inter-class discrimination Object detection (single class) 1,000β3,000 annotated images Anchor-based; more for multi-scale variation Segmentation 500β2,000 annotated images Pixel-level annotation is expensive; consider weak supervision Anomaly detection (good-only training) 200β500 good samples More robust with 1,000+; scale with visual complexity Training from scratch (no pretrained backbone) typically requires five to ten times these volumes. In our experience, most production projects underestimate the data requirement for edge cases and rare classes. Performance on common cases looks acceptable early in the project, and the edge-case failures only emerge under operational exposure β usually weeks or months after deployment, when the team has already moved on to other work. Data augmentation strategy Augmentation artificially expands training diversity and is one of the highest-leverage investments in training pipeline quality. But augmentation must be domain-appropriate. Applying the wrong augmentations degrades rather than improves model performance. Generally safe (almost always beneficial): Horizontal and vertical flips, where orientation is not semantically meaningful. Random crops and resizing. Brightness and contrast jitter within a moderate range. Gaussian noise and blur. Domain-specific (verify they match real variation): Rotation: beneficial if the deployment shows rotated objects; harmful if orientation is a class cue. Colour jitter: appropriate for scenes with variable lighting; inappropriate if colour is a discriminating feature. Cutout/random erasing: good for partially occluded objects; may hurt if full visibility is required. Use carefully: Aggressive geometric distortion can break texture-based features that matter to the task. Colour inversion or channel shuffle rarely matches real variation and often hurts. Synthetic mixing (CutMix, MixUp) is effective for classification but can confuse detection and segmentation models. Track augmentation strategy separately from model architecture in experiment logs. This is an observed pattern across our engagements: augmentation choices explain more performance differences across experiments than architecture choices do in most production image processing scenarios. The team that runs disciplined augmentation ablations usually arrives at a stronger model than the team that runs disciplined architecture ablations on a fixed augmentation pipeline. Deployment optimisation A model that runs at two seconds per image in a research environment must be optimised for production latency. The four standard levers, roughly in the order we apply them: Quantisation. Converting model weights from FP32 to INT8 reduces model size by 4Γ and typically increases inference throughput by 2β4Γ on compatible hardware, with accuracy loss of 0.5β2% for well-calibrated quantisation. INT8 quantisation requires calibration data β representative input samples β for activation quantisation. This is an observed range across image classification and detection workloads on NVIDIA and ARM targets; verify on your own held-out set. Inference runtime conversion. Exporting PyTorch or TensorFlow models to TensorRT or ONNX Runtime typically gives a 3β5Γ throughput improvement over native PyTorch inference on NVIDIA hardware for batch sizes of 1β16. This is usually the highest-leverage single step and should be measured, not assumed. Model pruning. Removing low-importance weights or channels. Structured pruning (removing entire channels) is hardware-efficient; unstructured pruning requires sparse hardware support. In practice, quantisation before pruning is usually the better path β quantisation gives most of the speed improvement with less accuracy risk. Model distillation. Training a smaller student model to match a larger teacherβs output distribution. Produces smaller models that approach the accuracy of the larger one. Useful when the production hardware cannot run the full model at required throughput even after quantisation. Deployment optimisation checklist Latency requirement defined (milliseconds per image or images per second). Target hardware specified (GPU model, embedded accelerator, CPU). Baseline inference time measured on target hardware before optimisation. INT8 quantisation accuracy validated on a held-out test set. ONNX export tested and validated (outputs match PyTorch within tolerance). TensorRT / ONNX Runtime throughput benchmarked on target hardware. Model size fits within the memory budget of the target device. Why does distribution shift quietly degrade production models? Distribution shift is the most insidious production failure mode. Model accuracy degrades gradually as the input distribution drifts away from the training distribution, but the degradation is not obvious without active monitoring. Unlike a crashed service or a failed deploy, a drifting model continues to return predictions β they are just increasingly wrong. Common sources: Camera hardware changes. A different camera model, lens, or mount position changes image statistics in ways the model was never trained to absorb. Lighting changes. Seasonal variation in natural light, replacement of lighting fixtures, repainted walls β all shift the scene illumination. Subject population changes. New product variants, new demographics, new defect types not present in the training set. Process changes. Changes in manufacturing process, retail layout, or operational workflow that change what the camera actually sees. Detection and response: Monitor confidence score distributions over time. A drop in average confidence without a corresponding change in labelled accuracy is an early warning sign. Monitor prediction class distributions. A shift toward edge classes or unusual class imbalance often indicates an input distribution change. Implement periodic validation against a fixed held-out test set β not just rolling production metrics, which can drift along with the input. When drift is detected, collect and label new samples from the current input distribution before retraining. Retraining on stale data simply re-creates the problem. In our experience, teams that build this monitoring into the deployment from day one detect drift early and respond with targeted retraining. Teams that deploy without monitoring discover drift only after users report degradation β typically months after the drift began, and after enough bad predictions have accumulated to erode trust in the system. The architecture decisions in the first half of this article β CNN versus ViT, quantisation, runtime conversion β determine how fast and how accurately the model serves a single image. The monitoring discipline in this section determines how long the answers remain trustworthy. Both belong to the same engineering job; neither is sufficient on its own. FAQ How do I architect a modular computer vision pipeline for production reliability? Treat the pipeline itself as the architecture decision, not the model. Each stage β ingestion, preprocessing, inference, post-processing β should be independently testable, replaceable, and observable. The model architecture decisions in this article (CNN vs ViT, quantisation, runtime conversion) sit inside the inference stage of that larger pipeline; the modular CV pipeline methodology covers the surrounding stages in full. What are the stages of a production CV pipeline, and which ones break first? The canonical stages are ingestion, preprocessing, inference, and post-processing. In our experience, the stages that break first under operational load are ingestion (camera or feed instability) and the preprocessing-to-inference boundary (resolution, normalisation, or colour-space drift between training and production). The model itself rarely fails first; it just produces increasingly wrong outputs because its inputs no longer match what it was trained on. How does vision-system integration differ between custom CV and off-the-shelf machine vision? Off-the-shelf machine vision (Keyence-style) ships a tightly coupled camera, lighting, and inference stack tuned for a narrow inspection task. Custom CV decouples those layers so each can be optimised independently β different cameras, different inference hardware, different models per task β at the cost of more integration work. The decision usually turns on whether the task fits the off-the-shelf vendorβs narrow envelope or requires the flexibility that decoupling buys. Where should pre-processing, inference, and post-processing live β same service or separate stages? Separate stages, each independently deployable. Same-service pipelines are faster to ship but harder to debug: when accuracy drops, you cannot tell whether preprocessing, the model, or post-processing is responsible. Separation lets you isolate the failing stage, replace it, and re-benchmark β which is the whole point of a modular pipeline. How do I make each pipeline stage independently observable and replaceable? Define a stable contract at each stage boundary (input schema, output schema, latency budget). Log inputs and outputs at the boundary, not inside the stage. Version each stage independently. When a stage needs replacement β a new model, a new preprocessing step β the contract is what guarantees the rest of the pipeline keeps working. What does a modular architecture buy me when a model needs to be retrained or swapped? A clean swap. The model becomes one stage with a defined contract; retraining or replacing it touches only that stage. Without modularity, model changes ripple through preprocessing assumptions, post-processing thresholds, and downstream consumers β and every change becomes a full-pipeline integration project.
