AI object detection looks like a solved problem on the leaderboard and an unsolved one in the warehouse. The architectures are mature, the pretrained weights are a pip install away, and a YOLO11 demo on COCO will happily draw boxes around anything you point a camera at. The trouble starts when that same model meets a fixed mounting angle, an LED panel that flickers at 50 Hz, and a class distribution that nobody measured before the camera went live. We see this pattern regularly: a team validates an off-the-shelf detector against a curated test set, ships it, and then discovers — sometimes weeks later — that the false-positive rate on the actual production stream is two or three times what the validation run reported. The model is not broken. It is simply being evaluated against a distribution it never saw. What “AI in object detection” actually means in 2026 A modern detector ingests an image or video frame and emits, per object, a bounding box, a class label, and a confidence score. The architectural lineage splits cleanly into three camps: Transformer-based detection — DETR, Deformable DETR, RT-DETR, DINO-DETR. Strong on complex scenes, less prone to anchor-tuning gymnastics. Real-time convolutional / hybrid — YOLO11, YOLOv10, YOLO12. Optimised for throughput; the default choice when latency is the binding constraint. Open-vocabulary — Grounding DINO, OWLv2. The class set is a text prompt rather than a fixed taxonomy, which changes how deployment is validated entirely. Adjacent to all three sits SAM-2, which does promptable segmentation and is increasingly used as a substitute for detection when shape matters more than category. The relevant point for production: choosing among these is not the hard part. The hard part is what comes after the choice. Where off-the-shelf detection breaks The benchmarks that train and rank these models — COCO, Open Images, LVIS — were never designed to predict deployment behaviour. They measure detection quality on a curated distribution under benign capture conditions. In our experience across CV engagements, four failure classes account for most production regressions: Lighting variability. Mixed natural and artificial light, glare on reflective surfaces, IR overspill at dusk, exposure drift across a 24-hour cycle. A model trained on web-scraped daylight imagery is structurally unprepared for any of this. Occlusion and pose distribution. COCO objects are mostly centred, mostly unoccluded, mostly photographed by humans who frame their shots. A ceiling-mounted camera over a retail aisle produces none of those properties. Class-distribution shift. The training set’s class frequencies will not match the deployment’s. Rare classes the model handled adequately in validation become invisible in production simply because the prior is wrong. Throughput and latency contracts that benchmarks never test. mAP at unconstrained inference time is not the same metric as mAP at 30 ms per frame on a Jetson Orin. Quantisation, batch shape, and pipeline overhead each shift the operating point. This is an observed pattern across our CV engagements, not a benchmarked rate — but the pattern is consistent enough that we now treat it as the default assumption rather than the exception. Production-versus-benchmark quick comparison Dimension Benchmark setting Production setting Lighting Curated, mostly daylight Mixed, time-varying, often artificial Camera framing Human-composed Fixed mount, often non-ideal angle Class distribution Balanced or natural web Skewed, long-tailed, drifting Latency budget Unconstrained 10–50 ms per frame typically Hardware Server GPU (A100, H100) Edge NPU, Jetson, or constrained CPU Evaluation metric mAP@COCO False-positive cost, miss rate per class Distribution drift None (static set) Continuous (seasonal, operational) The columns do not converge over time. They diverge as soon as the deployment encounters a condition the training distribution under-represented. How to test a CV model against production data before shipping The corrective discipline is not exotic. It is the willingness to do four things before, not after, deployment: Collect a representative validation set from the actual capture pipeline — same camera, same mount, same lighting cycle, same compression. A few thousand frames covering the operating envelope is usually enough to expose the dominant failure modes. Characterise failure modes per environment. Per-class confusion matrices, per-hour-of-day error rates, per-region-of-frame miss rates. Aggregate metrics hide the structural problems; stratified metrics reveal them. Set expected-performance contracts, not benchmark-accuracy claims. “False-positive rate below 0.5% on the night-shift stream” is a contract. “92% mAP” is a number that travels poorly. Run a shadow-mode period before the model affects any downstream decision. The cost is days; the alternative is discovering the gap with real consequences attached. When fine-tuning is enough versus when a replacement is needed comes down to whether the failures cluster around a few correctable axes (in which case targeted data collection and fine-tuning work) or whether they are diffuse and architectural (in which case the model class is the wrong fit). The diagnostic is the per-stratum error analysis above, not intuition about which model is “better”. Where AI object detection is deployed today Five high-volume categories carry most of the production tonnage: autonomous-driving and ADAS perception stacks; retail (loss prevention, shelf inventory, self-checkout); security and surveillance; industrial inspection and quality control; sports and broadcast analytics. A long tail covers robotics, agriculture, wildlife monitoring, and medical imaging. The hardware footprint runs from cloud GPUs down through Jetson-class edge servers to single-board NPUs (Hailo, Ambarella, Qualcomm), with quantised YOLO11n or NanoDet variants dominating the low-power end. What unites the five is not the model architecture — it varies — but the fact that the deployments that survive are the ones where the team validated against representative data and wrote performance contracts in operational terms. The ones that don’t survive are the ones where someone trusted the demo. For the broader argument about why off-the-shelf computer vision models fail in production, and for the engineering questions that sit one layer up, our Computer Vision R&D practice page is the entry point. Frequently asked questions Why do off-the-shelf computer vision models fail in production? Pretrained detectors are validated on curated benchmark distributions (COCO, Open Images, LVIS) that under-represent the lighting, framing, occlusion, and class skew of real deployments. They also rarely match the latency and hardware constraints of the production pipeline. The failure is structural, not edge-case: the evaluation distribution and the deployment distribution differ along several axes at once. What kinds of edge cases break public detection / classification models in real deployments? Four dominate in our experience: variable and time-varying lighting (mixed artificial / natural, glare, IR overspill); fixed-mount camera angles that produce framing the model never saw; class-distribution drift where rare-in-training classes become common-in-deployment or vice versa; and throughput constraints (10–50 ms per frame on edge hardware) that change the operating point relative to the benchmark mAP. How do I test a CV model against production data before shipping it? Collect a few thousand frames from the actual capture pipeline (same camera, mount, lighting cycle, compression), build stratified evaluation slices (per-class, per-hour, per-region-of-frame), and run the model against those slices before deployment. Then run a shadow-mode period where the model’s outputs are logged but do not affect downstream decisions. Aggregate metrics hide failure modes; stratified metrics surface them. What does it cost to discover an off-the-shelf model is wrong only after deployment? The cost shows up as false-positive remediation work (manual review, customer-facing errors, downstream system corrections) and as miss-rate consequences (undetected defects, lost inventory events, security misses). The expensive part is not the model rework — it is that the false-positive and miss rates were never measured under real conditions, so the operational team is debugging blind for the first weeks. When is fine-tuning enough versus replacing the model entirely? Fine-tuning works when failures cluster around a few correctable axes: a specific lighting condition, a specific class, a specific camera angle. Replacement is the right call when failures are diffuse — when the model is wrong across many strata and the per-stratum error analysis does not surface a dominant correctable cause. The diagnostic is the stratified evaluation, not intuition about which architecture is newer. Which object-detection problems are inherent to the model class versus solvable with more data? Resolution limits, real-time inference budgets, and open-vocabulary handling are largely architectural — they constrain what any amount of fine-tuning can achieve. Class imbalance, domain shift, and most failure modes tied to specific environments are data problems, addressable by collecting the right representative set. The line between the two is rarely obvious before the stratified evaluation is done, which is why the evaluation is the precondition for the architectural decision rather than the consequence of it.
