What does graceful degradation mean for a product recognition system? In a large-scale SKU recognition deployment we ran for a retail technology client, the system achieved 95.6% top-1 accuracy at 1,000 product classes (project-specific operational measurement). The same architecture, retrained and expanded to 2,000 classes, returned 83.5% — a 12-point drop that was not evenly distributed across the catalogue. That measurement is the seed of every architectural decision in this article: the question is not whether the drop happens but whether the system absorbs it gracefully or starts producing silent misclassifications that look acceptable on a dashboard and degrade operational outcomes for months before anyone notices. A product recognition system that degrades gracefully does not maintain constant accuracy as the SKU catalogue grows — no system does that without continuous retraining. What it does is maintain operational viability: a predictable, measurable automation rate with explicit handling for the cases it cannot resolve, rather than a growing pool of silent misclassifications. The distinction matters because the alternative — a system that maintains high aggregate accuracy through the first year of operation and then degrades unpredictably as the catalogue expands — is indistinguishable from a well-functioning system until the operational consequences appear. By that point, the architectural decisions that would have prevented the degradation have been locked in for eighteen months. This is the architectural response to the failure class. We have written separately about why computer vision fails at retail scale — that piece is the diagnosis lens. This article is what the architecture looks like once the team has accepted the diagnosis: given the failure class is real and predictable, what does the system look like that absorbs it gracefully? The degradation curve and why it matters architecturally The 95.6% → 83.5% drop above is not evenly distributed across the catalogue. The top-500 classes by training sample count maintained accuracy above 91% (operational measurement from that project). The bottom-500 classes degraded to 71% (operational measurement from that deployment). This distribution is the key architectural insight: the degradation is not uniform. It concentrates in the long-tail classes — the products with fewer training examples, higher visual similarity to adjacent classes, and more frequent visual similarity to newly added classes in the same category. A system designed for graceful degradation responds to this distribution differently from a system optimised for aggregate accuracy. Architectural choices for graceful degradation Class-specific confidence thresholds. A single global confidence threshold produces different false-positive rates for high-frequency and low-frequency classes. Setting per-class or per-category confidence thresholds — calibrated against the per-class accuracy on the validation set — allows high-confidence routing for the well-performing classes while applying more conservative thresholds to the long-tail. This converts some misclassifications into explicitly unresolved decisions that can be routed to review, rather than silent errors. Explicit retraining triggers. Rather than retraining on a fixed schedule, monitor per-class accuracy in production and trigger retraining when specific class accuracy drops below a defined threshold. This focuses the retraining investment on the classes that need it and avoids retraining the full catalogue when only a subset of classes has drifted. Catalogue expansion planning. Before each catalogue expansion cycle, estimate the per-class accuracy impact of the new classes on existing ones. New classes with high visual similarity to existing classes (same packaging format, similar colour profile) can be identified from the feature-space representation before they are added to the live model. This allows proactive data collection for classes that will create new decision boundaries, rather than reactive retraining after accuracy has already degraded. Unknown-object detection at expansion time. New SKUs added to the retail environment before they are added to the training catalogue are a source of misclassification that aggregate accuracy metrics cannot distinguish from correct classifications. An explicit out-of-distribution detector running alongside the classifier flags high-uncertainty predictions for review rather than returning a low-confidence classification. The unknown-object surfacing pipeline converts this source of silent error into an explicit review queue. The retraining loop design The retraining strategy for a growing-catalogue system determines whether the system improves continuously or requires periodic cold restarts. A cold restart — discarding the existing model and retraining from scratch on the full expanded catalogue — is operationally simple but expensive and breaks continuity. The system’s performance dips during each retraining cycle, and the production history (per-class accuracy trends, confidence calibration data, edge case examples) is discarded. An incremental retraining loop — adding new classes to the existing model using class-incremental learning techniques (experience replay, knowledge distillation from the previous model, elastic weight consolidation) — maintains performance on existing classes while adding capacity for new ones. The critical design parameter is the catastrophic forgetting rate: the speed at which the model loses accuracy on previously learned classes when trained on new ones. This rate is estimable before deployment and should inform the retraining frequency. For the SKU recognition deployment described above, an augmentation strategy combining synthetic data generation for visually similar classes with per-class hard-negative mining reduced the long-tail accuracy gap from 20 points to 11 points after one retraining cycle (project-specific outcome). The architectural decision that made this possible was having per-class accuracy monitoring in production — without it, the team would not have known which classes to prioritise. Class-incremental retraining: schedule and named techniques Class-incremental learning is the literature term for the problem of adding new classes to an already-trained classifier without retraining from scratch and without destroying performance on the existing classes. The retail SKU expansion problem is one of its cleanest practical instances. The schedule below is the structure we use; the specific technique selection depends on the catastrophic forgetting rate measured for the deployed architecture. Cadence. Trigger an incremental retraining cycle on either of two conditions — as a planning heuristic from our SKU-recognition engagements (not a benchmarked industry rate): (a) the cumulative new-SKU count since the last cycle reaches 5–10% of the existing catalogue, or (b) the per-class accuracy on any monitored class drops below the alerting threshold for two consecutive measurement windows. Time-based cadences (quarterly retraining) are inferior to data-driven triggers because they over-train when the catalogue is stable and under-train during expansion phases. Technique selection. Three families of class-incremental techniques are practical for production SKU recognition: Knowledge distillation from the previous model (Learning without Forgetting, LwF). The previous model serves as a teacher, and the new model is trained on a combined loss: standard cross-entropy on the new classes plus a distillation loss that keeps the new model’s logits close to the previous model’s logits on the same inputs. LwF requires no storage of historical training data, which makes it the lowest-friction option, and it is straightforward to implement on top of a PyTorch training loop. The trade-off is that LwF alone tends to drift on the hardest existing classes when the new-class count is large. Memory replay with herding-based exemplar selection (iCaRL). A small per-class memory buffer of exemplars (typically 20–50 images per class) is maintained across retraining cycles. The exemplars are selected by herding — picking the images whose features best approximate the class mean in feature space. During incremental training, exemplars are replayed alongside the new-class data. iCaRL outperforms LwF when memory budget permits, at the cost of maintaining the exemplar store and re-selecting exemplars after each cycle. Gradient Episodic Memory (GEM) and its variants. Constrains parameter updates so that the loss on stored exemplars from previous tasks does not increase. More expensive per training step than LwF or iCaRL but produces stronger forgetting resistance when the per-class budget is small. Worth considering when the deployment is on a hardware tier where the model cannot be enlarged to absorb the new classes purely additively. Validation gate. Before promoting a retrained model to production, validate it against a fixed historical test set whose composition does not change between cycles. The fixed test set is what makes per-cycle accuracy comparable. A second validation pass on a recent-data slice covers the new classes and any environmental drift. Rollback path. Each retrained model should be deployable alongside its predecessor for a defined evaluation window, with traffic split or shadow-mode comparison enabled. A retrained model that improves aggregate accuracy but degrades a specific high-value SKU class should be rolled back, not promoted, and the failure mode investigated. What remained imperfect The SKU recognition system described here met its operational targets, but two limitations were not resolved within the project scope and remain worth naming: First, the synthetic data generation step that closed part of the long-tail gap was domain-specific — it relied on photographic templates of pack formats that worked well for the dominant retail categories in the deployment but did not transfer cleanly to categories with high intra-class visual variation (fresh produce, bakery items packaged inconsistently). For those categories the long-tail accuracy gap remained closer to the original 20-point figure, and the operational handling relied on routing them to manual review rather than automating them. Second, the class-incremental retraining loop was effective for catalogue additions but did not fully solve the removal problem. When a SKU was discontinued, the model continued to recognise it for some time, occasionally classifying its successor product into the discontinued class. Cleaning up discontinued classes from the model required either a fuller retraining pass or an explicit “unlearning” step that we treated case by case rather than systematising. What graceful degradation looks like operationally A system with a well-designed degradation profile looks like this: as the catalogue grows, aggregate accuracy declines modestly and predictably. The automation rate on the well-performing class tier remains stable. The explicitly unresolved decision rate grows at a manageable pace proportional to the catalogue expansion rate and feeds directly into the retraining pipeline. Operators interact with a review queue whose volume they understand and can plan around, not a set of accuracy regressions they cannot explain. The alternative to this design is not a simpler system — it is a system where the same operational cost exists but is distributed invisibly across misclassifications, manual spot-checks, and customer complaints rather than explicit review queues. When computer vision is evaluated honestly for ROI in retail, the automation rate on the well-performing class tier, not the aggregate accuracy, is the number that determines whether the business case holds. A Production CV Readiness Assessment for retail evaluates a planned product recognition system against the architectural choices described here — confidence routing, retraining triggers, expansion planning, and unknown-object handling — before deployment, when the choices are still cheap to make.
