A tracking model does one job that a per-frame detector cannot: it keeps the same object attached to the same identity as it moves across frames. Run a detector on frame after frame and you get boxes; you do not get tracks. The step that turns a stream of independent detections into a set of persistent object IDs is association, and association is where most tracking deployments quietly fail — not because the tracker algorithm is weak, but because the geometry of the boxes feeding it is wrong for the scene. That distinction matters because tracking is usually treated as a solved add-on. The common plan is: run the detector, wire the boxes into an off-the-shelf tracker, assume identities follow. It holds right up until the association step meets the same geometry problem the detector already faces — rotated or densely packed objects, where an axis-aligned box captures background and overlaps its neighbours. At that point the tracker inherits the detector’s bad geometry, and identity switches start appearing exactly where objects sit close together. What is a tracking model, and what does it mean in practice? A tracking model associates detections across consecutive frames so that each real-world object receives a stable identity for as long as it stays in view. In practice it sits downstream of a detector. Each frame, the detector emits boxes with class labels and confidence scores; the tracker’s job is to decide which of this frame’s boxes correspond to which of last frame’s tracks, spawn new tracks for genuinely new objects, and retire tracks for objects that have left. Two cues drive that decision. The first is motion and overlap: an object that was here last frame is probably near here this frame, so a box that overlaps the predicted position of an existing track is a strong match candidate. The second is appearance: a learned embedding of the object’s pixels lets the tracker re-attach an identity even after a brief occlusion, when overlap alone would have broken. Classical trackers such as SORT lean almost entirely on motion and IoU; DeepSORT and its successors add an appearance embedding to survive occlusions. Either way, both cues degrade when the box does not fit the object — and that is the crux of this article. If you want the mechanics of the association loop itself — the predict-match-update cycle, Kalman filtering, the Hungarian assignment — we cover that in detail in how an object tracker works. Here the focus is narrower: why the quality of the detection geometry decides whether that loop produces stable IDs or a mess of switches. Detection versus tracking: where the boundary sits People conflate object detection with object tracking, and the conflation hides the failure mode. Detection answers “what is in this frame, and where” — independently, one frame at a time. Tracking answers “which of these things is the same thing I saw before” — a relationship across frames. A detector has no memory; a tracker is almost entirely memory. That division of labour is what makes the interface between them so consequential. The tracker cannot fix a detector’s mistakes about geometry — it can only associate the boxes it is handed. If those boxes are loose, tilted, or overlapping, the association math starts from a corrupted signal. The tracker is doing its job correctly and still producing wrong identities, because the input geometry was ambiguous before it ever reached the assignment step. For a fuller picture of how detection, classification, and re-identification embeddings all feed the association stage, see what feeds a tracker. The short version: garbage geometry in, unstable IDs out. How does association actually match detections between frames? The association step scores every pairing of a current detection against every active track, then solves the assignment. The dominant geometric score is Intersection over Union (IoU) — the overlap between a detection box and a track’s predicted box, normalised by their combined area. High IoU means “these are probably the same object.” Low IoU means “probably not.” The tracker builds a cost matrix from these scores (often combined with an appearance-distance term), and the Hungarian algorithm picks the globally best set of matches. Here is the mechanism that breaks in dense scenes. IoU is only a reliable signal when a box tightly encloses its object. Put an axis-aligned box around a rotated object — a pallet turned 30 degrees, a fish on a conveyor, a car photographed obliquely — and the box has to grow to contain the tilted extent. That inflated box now overlaps the boxes of neighbouring objects. Two adjacent objects can produce two inflated boxes with substantial mutual IoU, and the assignment step can no longer tell which detection belongs to which track. The result is an identity switch: object A’s track jumps onto object B’s box, or the two merge. The same inflation hurts one step earlier, during non-maximum suppression (NMS). NMS suppresses redundant boxes by comparing their IoU; when two genuinely distinct objects sit close and both boxes are inflated, NMS can wrongly treat them as duplicates and delete one. A false merge at NMS becomes a missing detection at the tracker, which becomes a fragmented track. This is precisely the problem that the detection head’s box-geometry decision determines — the choice between axis-aligned and oriented boxes is made upstream, and the tracker lives with the consequences. Why do axis-aligned boxes cause identity switches in dense scenes? Because the box is a proxy for the object, and a bad proxy corrupts every downstream comparison. Consider two rotated cartons packed side by side on a conveyor. An axis-aligned bounding box (AABB) around each one must include the diagonal extent, so a large fraction of each box is background — and much of that background belongs to the neighbour. The two AABBs overlap heavily even though the cartons themselves do not touch. To the association math, that overlap is indistinguishable from the overlap you would see if a single object had moved slightly between frames. The tracker cannot separate “two objects that are close” from “one object that moved,” and it resolves the ambiguity by guessing. In our experience with dense-packing and rotated-object deployments, this is the single most common source of identity churn — not occlusion, not motion blur, but geometry the tracker was never equipped to disambiguate (observed pattern across TechnoLynx computer-vision engagements; not a benchmarked rate). The fix is to give the tracker tighter geometry. How oriented bounding boxes tighten association An oriented bounding box (OBB) adds an angle to the box parameters, letting it rotate to hug the object’s true extent. A tightly fitted OBB around a tilted carton contains far less background and, critically, far less of its neighbour. The rotated-IoU between two adjacent OBBs is close to zero where the axis-aligned IoU was high. Association becomes decisive again: the assignment step can tell the two objects apart because their boxes no longer overlap in a way that lies about their spatial relationship. The payoff shows up in the tracking metrics. Where detections come from tightly fitted oriented boxes rather than inflated axis-aligned ones, IoU-based association is less ambiguous in dense scenes, which cuts identity switches and reduces the false merges that NMS would otherwise cause. The mechanics of angle regression and rotated-IoU are covered in rotated bounding box detection; this article’s point is downstream of that one — orientation is not just a detection nicety, it is what makes cross-frame association tractable when objects are packed. None of this is free. Oriented detection adds an angle-regression term to the detector, a rotated-IoU computation to NMS and association, and usually more demanding annotation. So the decision to feed oriented boxes into a tracker is a deployment-condition call, not a default. If your objects are upright and sparsely spaced, AABBs are cheaper and every bit as stable. The value of orientation is concentrated in scenes where orientation carries information and objects sit close together. Decision rubric: does your tracker need oriented detections? Score one point per row that describes your deployment. This is a planning heuristic drawn from repeated engagements, not a benchmark threshold. Condition Points if true Objects are frequently rotated relative to the camera axis +2 Objects are densely packed (neighbours within a box-width) +2 You already see identity switches concentrated in crowded regions +2 Orientation itself is a useful output (pose, alignment, defect angle) +1 Objects are upright and well-spaced −2 Annotation budget for angle labels is tight −1 Latency budget leaves no room for rotated-IoU cost −1 Score ≥ 4: oriented detection upstream of the tracker is likely worth the added cost — validate with an ID-switch A/B on your footage. Score 1–3: borderline; the axis-aligned baseline may already be adequate. Instrument first, decide from the numbers. Score ≤ 0: stay axis-aligned. Spend the budget on appearance embeddings or motion modelling instead. What metrics tell you a tracking model is working? Detection metrics like mAP do not tell you whether a tracker is stable, because they judge per-frame boxes, not cross-frame identity. Three metrics do the tracking-specific job: ID switches — how often a track’s assigned identity jumps from one object to another. This is the direct symptom of ambiguous association. If ID switches cluster in dense regions of the frame, box geometry is the first thing to check. Track fragmentation — how often a single object’s track is broken into pieces (lost, then re-acquired under a new ID). Fragmentation rises when NMS false-merges delete detections or when occlusion breaks appearance matching. Multi-object tracking accuracy (MOTA) — a composite that folds misses, false positives, and ID switches into one number. It is the headline figure, but you should always read it alongside the ID-switch count, because MOTA can look acceptable while a specific dense region churns identities. The right way to justify oriented detection is to quantify these three against the added angle-regression and association cost before committing. Measure ID switches and fragmentation on your own footage with an AABB baseline, then with oriented boxes, and let the delta — not the theory — make the call. Detection accuracy and tracking stability are related but not the same axis; if you need to ground the detection side of that comparison, object detection metrics explains what precision, recall, mAP, and IoU actually measure. When should tracking be part of the deployment at all? Not every video pipeline needs a tracker. If the downstream decision only asks “how many objects are in this frame” or “is there a defect here, now,” per-frame detection alone is simpler and has no identity to switch. Add tracking when the decision depends on an object’s history — counting unique objects crossing a line, measuring dwell time, associating an action with a specific entity, or smoothing detections over time to suppress single-frame flicker. Once you commit to tracking, you have committed to the association problem, and the association problem is where box geometry becomes load-bearing. That is why the feasibility of tracking and the feasibility of the detection geometry beneath it have to be assessed together, not in sequence. On a conveyor, for example, the box-geometry call that governs [multi-object tracking on a production line](manufacturing and automotive) is the same one the detector faces — and scoping them separately is how teams discover, in production, that a tracker they signed off on cannot hold identities in the packed regions that matter most. The broader engineering context for detection-to-tracking pipelines lives on our computer vision practice page. FAQ What should you know about a tracking model in practice? A tracking model sits downstream of a detector and associates the detector’s per-frame boxes so that each real-world object keeps a stable identity across frames. In practice it runs a predict-match-update loop each frame, using motion and box overlap plus (optionally) an appearance embedding to decide which current detection belongs to which existing track, spawning and retiring tracks as objects enter and leave. What is the difference between object detection and object tracking across frames? Detection answers “what is in this frame and where,” independently for each frame, with no memory. Tracking answers “which of these things is the same thing I saw before,” a relationship across frames that is almost entirely about memory. The tracker cannot correct a detector’s geometry errors — it can only associate the boxes it is handed. How does a tracking model associate detections between frames — what role do IoU and appearance cues play? Association scores every detection-to-track pairing and solves the assignment globally. Intersection over Union (IoU) measures box overlap and supplies the geometric match signal, while an appearance embedding supplies a re-identification signal that survives brief occlusions. Both cues degrade when the box fits the object poorly, because a loose box produces misleading overlap and diluted appearance features. Why does poor bounding-box geometry cause identity switches in dense scenes? An axis-aligned box around a rotated object has to grow to contain the tilted extent, so it fills with background and overlaps its neighbours. That inflated overlap makes two adjacent objects look, to the IoU math, like one object that moved — an ambiguity the assignment step resolves by guessing, which produces an identity switch. The same inflation causes NMS to false-merge distinct objects, fragmenting tracks. How do oriented bounding boxes improve tracking association in packed or rotated-object deployments? An oriented bounding box adds an angle so it hugs the object’s true extent, containing far less background and far less of its neighbour. The rotated-IoU between two adjacent oriented boxes drops close to zero where the axis-aligned IoU was high, so association becomes decisive again — cutting identity switches and the NMS false merges that fragment tracks. What metrics tell you whether a tracking model is working? ID switches count how often a track’s identity jumps between objects and are the direct symptom of ambiguous association. Track fragmentation counts how often one object’s track breaks into pieces. Multi-object tracking accuracy (MOTA) folds misses, false positives, and switches into one composite — read it alongside the raw ID-switch count, since MOTA can look fine while one dense region churns identities. When should tracking be part of the deployment versus per-frame detection alone? Use per-frame detection alone when the decision only needs the current frame — counting objects now, or flagging a defect now. Add tracking when the decision depends on an object’s history: unique counts across a line, dwell time, action attribution, or temporal smoothing. Once you add tracking you own the association problem, so assess it and the detection geometry beneath it together. If your identity switches cluster in the crowded parts of the frame, do not reach for a different tracker first — check the geometry feeding it. A tracker inherits the box quality it is given, and the divergence point between a stable pipeline and a churning one is almost always the detection geometry, not the association algorithm. That is the boundary the A2 production-geometry assessment is built to surface before it reaches the tracking stage.
