Plain accuracy is the wrong promotion gate for near-identical colourway detection. It counts every error as equally bad, and on this task the two kinds of error are not equally bad. A false pass, where the pipeline accepts a navy trainer as its midnight-blue sibling, and a false fail, where it rejects a correct match and sends it to manual review, cost different amounts. The gate should reflect that. We would gate on recall@k, measured on a purpose-built set of colourway-confusable images, with k and the minimum value set by what each error costs downstream. Embedding margin can serve as a secondary check.

Why does accuracy mislead when a false pass and a false fail cost different amounts?

Accuracy weights every sample equally. On imbalanced data that makes it insensitive to how the model does on the minority classes. Google’s Machine Learning Crash Course gives the general rule: when the dataset is imbalanced, or one kind of mistake costs more than the other, optimise for a metric other than accuracy. Suppose most queries are easy (a boot against a sandal) and a small share are hard (one style sold in six colourways). The pipeline can then post a high accuracy figure that tells you almost nothing about the hard share.

The principled replacement is expected cost. The paper Analysis and Comparison of Classification Metrics defines expected cost as a generalisation of the error rate for cases where errors do not all have equally severe consequences. It shows that the standard and balanced error rates are both special cases of it. Elkan (IJCAI 2001) states the decision rule that follows: predict the class with the lowest expected cost, computed from the conditional probability of each class given the example.

For a binary accept-or-reject decision with trustworthy posteriors, that rule reduces to a threshold that depends only on the costs. Accept the match when its posterior exceeds c12 / (c12 + c21), where c12 is the cost of a false pass and c21 the cost of a false fail. Take an assumed input: the merchandising team judges a false pass four times as costly as a false fail. The threshold is then 4 / (4 + 1) = 0.8, against 0.5 when both errors cost the same.

Two conditions decide whether that formula can be used directly:

• The posteriors must be calibrated and matched to the priors the system will see in deployment. Otherwise the Bayes decision is not optimal.
• If the scores are poorly calibrated, it is valid to skip the formula and pick the threshold that minimises the cost of interest on development data. The price is retuning it whenever the cost matrix or priors change.

Because embedding pipelines output cosine similarities rather than posteriors, we would take the second route for colourway matching.

The choice of metric for setting that threshold matters too. An empirical study of threshold selection for neural-network monitors (Can we Defend Against the Unknown?) began by optimising thresholds for F1. Under class imbalance, F1 set the threshold so low that every sample was rejected in 116 of 864 experiments. The study found that g-mean (the geometric mean of recall and specificity) is unaffected by class imbalance. Its broader conclusion was that the right measure depends on whether the system’s objective is fewer missed errors or fewer false rejections. Read across to colourways, that objective is the cost matrix written down.

Bach and colleagues’ paper Considering Cost Asymmetry in Learning Classifiers adds a point about training. When the cost asymmetry at test time is extreme, the asymmetry used during training should be less extreme. So the full cost ratio belongs at the decision threshold, not hard-coded into the loss weights. ROC curves are the standard display when end users’ costs are unknown at training time. Our piece on where cost-weighted ROC analysis applies and where it doesn’t covers when that tool holds up for setting a defensible threshold.

A recall@k protocol for colourway-confusable images

To measure whether a detection and embedding pipeline separates near-identical colourways, use a retrieval protocol rather than a classification score. Embed each query image, rank a gallery that deliberately includes the other colourways of the same style, and count how often the correct colourway appears in the top k. Promote a candidate only when that figure clears a threshold derived from the cost matrix, measured on a held-out set built for the purpose.

LynxBenchAI covers the hardware side. It benchmarks throughput, latency and memory across accelerators, which tells a team whether the device a passing pipeline will ship to can hold its throughput budget.

Recall@k and CMC@k are not the same number

The STIR authors (2023) point out a gap between classical recall and the “recall” reported in much of the metric-learning literature. True recall@k is the number of ground-truth results in the top k divided by the total number of ground-truth results. So when a query has several correct answers, recall@1 cannot exceed one over that number. Many papers report recall@1 values close to 1 anyway, because what they are actually reporting is the cumulative matching characteristic: CMC@k is 1 if any correct answer appears in the top k, and 0 otherwise.

When a colourway gallery holds several photos of each colourway, the two definitions give different numbers. The gate has to say which one it uses. The downstream action decides:

• Automatic top-1 acceptance (listing, routing or deduplication with no human in the loop): gate on CMC@1.
• A reviewer shown a shortlist: gate on CMC@k, with k equal to the shortlist length the reviewer actually sees.
• Grouping every image of a colourway (catalogue clean-up, variant clustering): gate on true recall@k, and track mean average precision alongside it, because mAP keeps registering ranking improvements after CMC@k has plateaued.

Widening k is not free. In a benchmark of pretrained embeddings for near-duplicate detection in 3D medical images, moving from top-1 to top-3 retrieval raised sensitivity and lowered specificity, because it let in more false positives. Take k from the downstream action, never from whichever value makes the candidate look best.

How do we set the recall@k threshold a candidate must clear before production?

Derive the threshold from the cost matrix and the downstream action, then check it against the uncertainty in the measurement. A round number chosen because it sounds safe is not a threshold. The steps run in this order:

1. Fix the action and k. Decide which of the three cases above applies and what k the action uses.
2. Write the cost matrix, including abstention. A cost matrix can include an “abstain” column, which here means sending the item to human review.
3. Turn the business requirement into a cost-based target. Where the application imposes a minimum sensitivity, sweep the cost weight. For each value, pick the threshold that minimises normalised expected cost on development data, then keep the weight that reaches the target sensitivity.
4. Tune on a development split, never the evaluation split. Using the evaluation set as the development set produces optimistic results.
5. Gate on the interval, not the point estimate. Compare the candidate’s confidence interval with the gate and with the incumbent. When two intervals for independent means just touch at their endpoints, that is roughly equivalent to a significant difference at p ≈ 0.01.
6. Check embedding margin as a secondary signal. Look at the gap between a query’s similarity to its nearest correct colourway and to its nearest confusable one. We would never let margin gate on its own.

Gating on the retrieval metric itself, rather than on a falling training loss, is deliberate. Lu and colleagues (ICCV 2019) argue, with a toy example, that when batches are far larger than k, conventional sampling has little chance of drawing examples near the top-k decision boundary, so the gradient may improve the embedding without directly improving top-k precision. Patel, Tolias and Matas (CVPR 2022) note that proxy losses can be poorly aligned with the evaluation metric, because recall is non-differentiable.

How large the confusable-pairs set has to be

The set has to be large enough that the recall@k estimate holds steady under resampling, and that its confidence interval is narrower than the distance between the candidate and the gate. A 2014 EVIA workshop paper gives a sense of how wide intervals can get. One TREC run had a mean average precision of 0.2175 with a standard error of 0.0343, which gives a 95% interval of 0.1491 to 0.2859. Size the set with a power analysis for the smallest recall difference that would change the promotion decision. If labelled data is short, average over repeated random splits; Bach and colleagues use 10 or 25 such splits for ROC estimates.

Size alone is not enough. If the set is small or unrepresentative of the real catalogue, the estimates will be unstable or optimistic. It should include the styles with the most colourways, the shades that photograph alike under store or studio lighting, and the image conditions the pipeline will see in production. New colourways that arrive after the set is built are not covered by it. The set needs a scheduled refresh, and the same discipline applies when the detector, camera or inference stack changes.

Which embedding training approach works best for near-identical colourways?

In embedding learning, the loss function does not decide the outcome on its own. How training examples are sampled plays an equal or more important role than the loss, and many published comparisons rest on flawed methodology. We would start from a detector that crops the product, followed by a metric-learning head fine-tuned from an open pretrained embedding model on the catalogue’s own images. We would let the batch size the hardware allows drive the loss choice, and check the result against the recall@k gate.

TechnoLynx builds this kind of hybrid detection and embedding pipeline through its Computer Vision R&D work. The first step is pinning down the architecture and the recall@k target for the catalogue. The loss and detector family then follow from the data and the latency budget.

Treat loss-function league tables with care. Musgrave, Belongie and Lim (2020) found flaws in the experimental methods of many metric-learning papers. They showed that real improvements over time had been marginal at best. Wu and colleagues (ICCV 2017) found that different sampling strategies produce very different solutions for the same loss, and that contrastive loss performs almost as well as triplet loss when both use the same sampling. So compare candidates under one protocol, with the same backbone, the same training budget and the same confusable test set.

Triplet loss with batch-hard sampling

Triplet loss pulls an anchor towards a positive and pushes it away from a negative. Its success depends on the triplets you choose. In a 2018 image retrieval study, uniform sampling performed worse than the pretrained baseline, so the authors adopted batch-hard sampling. On Stanford Online Products, with most batches drawn from one category, the same study found that larger batches helped up to about 32 anchor-positive pairs, with no significant gain beyond 48. In the colourway task itself, Myntra’s colour-variants study trained a supervised triplet model on manually annotated colour-variant labels, with a margin of 0.2 on a ResNet34 encoder. For its self-supervised SimSiam embeddings, skipping l2 normalisation left some data splits with no true colour-variant groups at all. A ViT backbone with plain triplet loss is still a competitive baseline for retrieval.

Google’s deep ranking work (2014) found that its multi-scale ranking model learned filters capturing more colour information than filters learned in earlier work. Our reading is that generic pretrained features can underweight exactly the signal a colourway task depends on.

ArcFace-style margins and classification losses

ArcFace adds an angular margin to the target angle on a normalised hypersphere, which gives a constant linear margin. SphereFace and CosFace give non-linear ones. In its face-recognition experiments it used a feature scale of 64 and a margin of 0.5. There, triplet loss reached similar intra-class compactness but weaker inter-class separation. Those results are on faces, and carrying them over to products is a judgement, not a finding.

For products, we would look first at batch size. A study of training choices for image retrieval found that classification losses such as CosFace and ArcFace do much better when the batch size is below 256. Contrastive losses overtake them at around 256 to 512. The same study notes that the classifier layer needs a far higher learning rate than the backbone being fine-tuned. Separately, normalised softmax has been shown to be a strong baseline for metric learning. Proxy-based losses avoid the pair-mining bottleneck altogether by learning shared class representatives.

Hard negative mining when the hardest negative is the sibling colourway

For colourways, the hardest negative is by construction the same style in another colour. Datasets with high intra-class and low inter-class variance produce hard triplets. With triplet loss, mining the hardest negatives often collapses the model so that every image gets the same embedding. Semi-hard mining avoids the collapse but works on a narrow band of examples, and progress stalls once that band empties. Two published fixes help. Xuan and colleagues (ECCV 2020) decouple hard triplets and penalise only the anchor-negative similarity, which makes optimising on hard negatives feasible. Wu and colleagues propose distance-weighted sampling, which lowers gradient variance whatever the loss. There is one more risk: pushing hard on the hardest negatives also amplifies mislabelled ones. On a colourway set, a label audit of the closest negative pairs is cheap insurance.

Embedding approaches compared for colourway work

Approach	Fits when	Watch for
Triplet loss with batch-hard sampling	Pair or triplet labels exist and batches hold at least about 32 anchor-positive pairs	Collapse with the hardest negatives; uniform sampling below the pretrained baseline
ArcFace or CosFace-style classification loss	Each colourway has a class label and batches are below about 256	Classifier layer needs a much higher learning rate; published margins were tuned on faces
Contrastive or multi-similarity loss with an online miner	GPU memory allows batches of roughly 256 to 512 or more	Classification losses do better at smaller batches
Recall@k surrogate loss	Very large batches, up to about 4k images, are possible	Benefit tied to batch scale; similarity mixup helps at small batches

Can we fine-tune an open embedding model on our own catalogue images?

Yes. Unless the catalogue’s imagery is very unusual, fine-tuning is the route we would prefer over training from scratch. A 2025 benchmark of image embeddings across six e-commerce datasets (fashion, consumer goods, cars, food and retail, none of them colourway sets) concluded that dataset-specific fine-tuning is necessary for optimal retrieval. It also found that top-tuning, which trains a few small layers on frozen embeddings, often matches full fine-tuning for self-supervised and text-image embeddings while using far fewer resources. On self-supervised pretrained models, top-tuning improved results by 5% on average. That makes top-tuning the cheap first experiment. The 2025 retrieval-training study found the best pipeline used DINOv2 CLS features with the backbone’s layers fine-tuned at a learning rate of 1e-6.

One vendor, Width.ai, reports that a fine-tuned CLIP model reached 89% on retail SKU matching against 41% for off-the-shelf alternatives. In our reading, that vendor result points the same way as the academic benchmark. Whether fine-tuning closes the gap on a given catalogue is an empirical question. The recall@k gate is what tells you whether it has.

How much labelled data per SKU or colourway?

The amount per colourway has to be found empirically, not assumed. As points of scale: Myntra’s in-house team annotated 6k images with colour-variant labels for a single garment category. Products-10K holds nearly 10,000 fine-grained SKUs in about 150,000 images, with a label noise rate below 0.5%. Neither number is a requirement. We would use a learning curve. Train on growing fractions of the labelled data, measure recall@k on the fixed confusable set each time, and stop labelling when the curve flattens inside its confidence interval.

Detector errors set the ceiling

A missed or badly cropped product cannot be retrieved correctly, however good the embedding is. So the detector’s recall caps what the whole pipeline can reach. Myntra’s pipeline used YOLOv2 to detect products before embedding them. We would choose a YOLO-family detector where throughput is tight, and a two-stage detector such as Faster R-CNN where scenes are cluttered and accuracy matters more than speed. We would measure the detector on its own and then run the gate end to end on uncropped images.

Who can build and validate a colourway-matching pipeline for our catalogue?

TechnoLynx, through its Computer Vision R&D work, builds a colourway-matching pipeline for your catalogue and validates it against a recall@k gate on your own confusable images, in these steps:

1. Frame the detection and embedding architecture and the recall@k requirement.
2. Choose the loss and the detector family to suit your data and latency limits.
3. Implement and fine-tune the pipeline on the target hardware.
4. Validate it with recall@k and embedding margin on the confusable-pairs set.
5. Deploy and operate it in your production environment.

Whether to bring in a specialist depends on whether your team already runs metric-learning training and a labelling pipeline. Whoever you choose, ask them three things: which recall definition they report, what interval sits around it, and which of your own images it was measured on.

This engagement is not for every catalogue. If your colourways are visually distinct, frozen pretrained embeddings may be enough, and a full build would be more than you need. A team with metric-learning experience can run this work with open tooling such as PyTorch Metric Learning. Where only the confusable-pairs set is missing, a labelling service is the more proportionate route.

The gate has to be re-earned

A recall@k gate is only as current as the costs and the catalogue it was set against. If the business changes how much a false pass costs, the threshold needs retuning, because a cost-specific threshold is only valid for the cost matrix it was tuned on. If a season adds colourways the confusable set has never seen, the measured recall@k no longer describes the pipeline in production. The question that remains open is not a modelling one: what does a wrong colourway actually cost you, compared with a review queue? Engineers can turn that ratio into a threshold. The business has to supply it.

FAQ

What is the difference between precision and recall in machine learning?

Recall is the share of actual positives the model correctly identifies, TP / (TP + FN). Precision is the share of positive predictions that are actually positive.

When should recall be prioritised?

Prioritise recall when false negatives cost more than false positives. Use the false positive rate when false positives cost more.

What is deep metric learning?

Deep metric learning learns a function that maps data samples into an embedding space where distance reflects similarity. It relies on sampling strategies and loss functions designed to keep confusable classes separable.