A computer vision model that hit 0.91 mAP three weeks ago just regressed in production, and nobody can rebuild the run that produced it. The checkpoint is named model_final_v3_actually_final.pt. The accuracy lives in a spreadsheet cell. The data split, the augmentation seed, the exact commit, the CUDA version — none of it was written down. Someone has to answer “what changed?” and the honest answer is: we don’t know, and we can’t find out without re-deriving a week of work. That is the failure experiment tracking exists to prevent. It is not a research luxury or a dashboard for showing off learning curves. It is the discipline that makes a training run reconstructible on demand — and for a computer vision team, reconstructibility under pressure is the difference between shipping a component in weeks and re-deriving lost work for quarters. What ml experiment tracking actually is Strip away the tooling and experiment tracking is one idea: treat every training run as a versioned artifact, not an event that happened once and is gone. A run is not just “the time we got 0.91.” It is a bundle — the code commit, the data snapshot, the hyperparameters, the runtime environment, and the metrics — captured together so the whole thing can be rebuilt later. The naive alternative is intuitive, which is exactly why it survives longer than it should. You keep a spreadsheet of accuracy numbers and name your checkpoints by hand. It works for one person running experiments serially. It collapses the moment two people train in parallel, or the moment you need to reproduce a result from three weeks ago and discover the inputs were never recorded. The divergence point isn’t the number of experiments. It’s reproducibility under pressure. A tracked run answers “what changed between the model that worked and the one that regressed?” in hours. An untracked run answers it in days of forensic reconstruction, if at all. That gap is the entire value proposition, and it only shows up when something goes wrong — which is why teams under-invest until the first bad incident teaches them. What should you log per training run to make a CV result reproducible? “Log the accuracy” is where beginners start and where most tracking stops. Accuracy tells you what happened. It tells you nothing about how to make it happen again. Reproducibility requires logging the inputs, not just the output. For a computer vision run specifically, the reconstructible set breaks into five parts: Category What to capture Why it bites if you skip it Code Git commit hash of the exact training code A one-line change to the loss or NMS threshold silently shifts results; without the hash you compare apples to a different tree Data Dataset version/snapshot ID, train/val/test split, label revision CV results are dominated by data; a relabeled batch or a changed split moves mAP more than most model tweaks Config Hyperparameters, augmentation pipeline + seeds, image resolution, class map Augmentation and resolution are first-class variables in vision; an unlogged seed makes a run unrepeatable by construction Environment Framework version (PyTorch/TensorFlow), CUDA/cuDNN version, container image cuDNN and TensorRT kernel selection can shift numerics; “it worked on my machine” is a reproducibility bug, not a joke Outputs Metrics over time, final checkpoint, confusion/PR curves, sample predictions The result itself, tied to everything above so the checkpoint is never an orphan The rule of thumb: if you can’t rebuild the run from what you logged, you didn’t track it — you just observed it. The data snapshot is the part CV teams neglect most and regret hardest, because vision performance is dominated by the dataset far more than by architecture choices. If you want the full picture of which metrics matter for detection specifically, we go deeper in object detection metrics explained: precision, recall, mAP and IoU — tracking is what preserves those numbers against a reconstructible baseline. How tracking connects code, data, and model artifacts The mechanism that makes reconstruction possible is linkage. A tracker doesn’t just store five separate piles of information — it binds them under a single run ID so that a checkpoint always points back to the exact code, data, and config that produced it. In practice this works through a small set of primitives. A run is the atomic unit — one training invocation with a unique ID. Parameters are the inputs you set before training (learning rate, batch size, backbone). Metrics are the time series you emit during training (loss, mAP per epoch). Artifacts are the files a run produces or consumes — the checkpoint, the config file, sometimes the dataset manifest. Tags and lineage links tie a run to its git commit and data version. The data side often needs its own versioning layer, because datasets are too large to live in git. A common pattern is to pair a metrics tracker with a data-versioning tool: the tracker records that run 4a7f used dataset snapshot v2.3, while the data tool records what snapshot v2.3 actually contains and lets you check it out byte-for-byte. Together they close the loop. Without the data link, you have a beautiful record of hyperparameters attached to a dataset you can no longer reproduce — which is the most common way “we tracked everything” still fails. TensorBoard sits at the lightweight end of this spectrum: it visualizes metrics and some artifacts well but doesn’t version code or data for you. We cover what it does and doesn’t handle in TensorBoard logging for computer vision pipelines: what to track and why. Knowing where a tool stops is as important as knowing what it does. Which experiment tracking tools fit which team size? There is no single correct tool — the fit depends on team size, whether your data lives in git-scale or dataset-scale storage, and how much of the stack you want to self-host. The three most common choices for CV teams pull in different directions. Tool Best fit Strengths What it doesn’t do alone MLflow Small-to-mid teams wanting open-source, self-hosted control Tracking + model registry, framework-agnostic, no vendor lock-in Data versioning; you pair it with a data tool Weights & Biases Teams that want zero-setup hosted tracking + rich CV visualization Excellent image/prediction logging, sweeps, collaboration UI Fully self-hosted control (SaaS-first; on-prem is enterprise) DVC Teams whose bottleneck is data and pipeline reproducibility Git-style versioning of datasets and pipeline stages Rich metric dashboards; it complements a metrics tracker The important thing is that these are not strictly competitors. A frequent production pattern is DVC for data and pipeline versioning + MLflow or Weights & Biases for metrics and the model registry — one tool owns the data lineage, the other owns the run lineage, and a run ID stitches them. Picking “the best tool” is the wrong framing; picking the two tools that cover data-versioning and run-tracking without overlap is the right one (observed pattern across engagements; not a benchmarked ranking). For a beginner CV team, the honest answer is to start with whatever imposes the least ceremony and captures the five categories above. Weights & Biases gets you logging in an afternoon. MLflow gives you more control if you already run your own infrastructure. You can migrate later; you cannot recover a run you never logged. What is the minimal experiment tracking setup a beginner CV team needs? The minimal viable setup is smaller than most tutorials suggest. You do not need a full MLOps platform to ship your first production component. You need four things wired together: A metrics/params tracker in the training loop. A dozen lines of MLflow or Weights & Biases calls that log hyperparameters at start and metrics per epoch. This is the lowest-effort, highest-leverage change. A git commit per run. Never train from a dirty working tree you can’t identify. Log the commit hash with the run; the code becomes reconstructible for free. A named, immutable data snapshot. Even a manually versioned folder or a DVC-tracked directory. The rule is that the training data for a logged run can never be silently mutated. The environment pinned. A requirements.txt or a container image tag logged with the run, so PyTorch and CUDA versions are recorded. That is the whole starter kit. It is roughly a day of setup and it eliminates the entire class of failures where a strong result cannot be recovered because its inputs were not logged. Everything past this — sweeps, model registries, automated promotion — is refinement you add once the base habit holds. This kind of setup discipline is exactly the habit we teach when onboarding practitioners into a production computer vision workflow, and it’s the same discipline that carries through to shipping, which we cover in ML model deployment tools: shipping classical and deep CV stages to production. How disciplined tracking shortens the path to a first shippable model The ROI is not the dashboard. It is compounding. Untracked experiments evaporate — each run’s learning is trapped in someone’s memory and lost when they move on or simply forget which config gave the good number. Tracked experiments accumulate: every run adds a comparable, reconstructible data point to a growing baseline. For a beginner entering CV from an adjacent field, this is what converts a pile of one-off notebooks into an auditable pipeline. Teams with disciplined tracking cut model reproduction time from days to hours (observed pattern; not a benchmarked rate), and — more importantly — never hit the wall where a promising result simply can’t be rebuilt. That directly supports the goal of getting a first-shippable component out in weeks rather than quarters, because the work you did last Tuesday is still usable on Friday. The failure this prevents is quiet until it isn’t. Nothing breaks the day you skip logging the data snapshot. It breaks three weeks later, in production, under time pressure, when “what changed?” has no answer. FAQ How does ml experiment tracking work in practice? Experiment tracking treats every training run as a versioned artifact rather than a one-off event. In practice it means capturing the code commit, data snapshot, hyperparameters, environment, and metrics together under a single run ID, so any past result can be reconstructed on demand instead of re-derived from scratch. What should you actually log per training run — beyond just accuracy — to make a CV result reproducible? Log five categories: the git commit for the code, the dataset version and split for the data, the hyperparameters and augmentation seeds for the config, the framework/CUDA versions for the environment, and the metrics plus final checkpoint for the outputs. Accuracy alone tells you what happened but not how to repeat it; the data snapshot is the part CV teams neglect most and regret hardest. Which experiment tracking tools (MLflow, Weights & Biases, DVC) fit which team size and CV workload? MLflow suits small-to-mid teams wanting open-source self-hosted control; Weights & Biases suits teams wanting zero-setup hosted tracking with rich CV visualization; DVC suits teams whose bottleneck is data and pipeline reproducibility. They are often combined — DVC for data lineage plus MLflow or W&B for run tracking — rather than chosen as strict competitors. How does experiment tracking connect code versions, data snapshots, and model artifacts so a result can be reconstructed later? A tracker binds everything under a run ID: parameters, metrics, and artifacts link back to the exact git commit and data version that produced them. Because datasets are too large for git, a data-versioning tool records what each snapshot contains while the tracker records which run used it — together they let you check out and rebuild the run byte-for-byte. What is the minimal experiment tracking setup a beginner CV team needs to ship a production component? Four things wired together: a metrics/params tracker in the training loop, a logged git commit per run, a named immutable data snapshot, and a pinned environment (requirements file or container tag). That is roughly a day of setup and it eliminates the class of failures where a strong result cannot be recovered because its inputs were never logged. How does disciplined tracking shorten the path from first experiment to first-shippable model? Tracked experiments compound instead of evaporating — every run adds a comparable, reconstructible data point to a growing baseline, so earlier work stays usable. This cuts model reproduction time from days to hours and turns a pile of one-off notebooks into an auditable pipeline, supporting a first-shippable component in weeks rather than quarters. When does a spreadsheet stop being enough, and what breaks first as a CV team scales? A spreadsheet works for one person running experiments serially. It collapses the moment two people train in parallel or you need to reproduce a result weeks later. What breaks first is reproducibility under pressure: when a model regresses in production and you must answer “what changed?” in hours, the untracked inputs simply aren’t there to compare. The question worth sitting with isn’t which tool to adopt. It’s whether, right now, you could rebuild your best-performing model from what you’ve written down — and if the honest answer is no, that’s the failure class to close before it closes on you in production.
