An on-call engineer gets paged at 3 a.m. by an anomaly model that has started flooding the channel. The model script is in Git, the commit hash is clean, and yet nobody can say what changed — because the thing that changed was never the code. Someone retuned the sensitivity threshold two weeks ago, the training window rolled forward on a schedule, and the false-positive rate crept past the point the on-call rotation can absorb. This is the failure that most teams running operational anomaly detection walk into: they version the training script and stop there. Machine learning version control, in practice, is not a Git repository for your model code. It is the discipline of versioning the data slice, the trained model artefact, and the tuning configuration together — as one reproducible state — so that any alert an engineer sees can be traced back to exactly the model, data window, and thresholds that produced it. Get that link right and a bad retune becomes a diff and a rollback. Get it wrong and you are guessing under load. Why “version the code” quietly fails in production The intuition is reasonable. Code is what you write, code lives in Git, so versioning the model means versioning the code. That works for deterministic software. It does not work for a system whose behaviour is a function of three things the code does not fully capture: the data it was trained on, the weights that training produced, and the operational knobs someone turned afterward. Consider what actually determines whether an anomaly model fires. A well-known isolation-forest or autoencoder detector will behave completely differently depending on the window of telemetry it was fit against — a month that happened to include a compressor trip teaches the model a different notion of “normal” than a quiet month did. The trained artefact is not reproducible from the script alone unless you also pin the data slice, the random seed, and the library versions. And the sensitivity threshold — the one number that decides how many alerts reach a human — frequently lives outside the model entirely, in a config file or, worse, in someone’s memory. So when the alert-flood erupts, the team that versioned only code has a clean commit history that explains nothing. They cannot answer the one question that matters: what state was in production when the behaviour changed? This is the core reason spec-clean version control misleads operators — the reproducible unit is the data-plus-model-plus-config triple, not the source file. It is the same class of gap we see when memory-intensive anomaly-detection workloads are sized against a script’s declared footprint rather than the runtime state that actually loads. What you actually version in an anomaly-detection deployment The practical answer is four artefacts, versioned so that they resolve to a single reproducible state. Each answers a different question when something breaks. Artefact What it captures The question it answers under load Where it usually lives Training / inference code The model logic and feature pipeline “Did the algorithm change?” Git Data slice The exact telemetry window + preprocessing the model saw “Did the definition of normal shift?” DVC / LakeFS / object-store snapshot + manifest hash Model artefact The trained weights or fitted detector “Is this the model that was live?” Model registry (MLflow) / hashed artefact store Tuning configuration Sensitivity thresholds, alert-suppression rules, window length “Did someone change the knobs?” Versioned config, pinned to the model release — not a loose file The last row is where most industrial and energy teams lose reproducibility. The tuning configuration is the highest-leverage, lowest-tracked artefact in the whole stack. A one-line change to a threshold shifts the false-positive rate directly against the on-call bandwidth limit, and if that change is not versioned alongside the model release it references, you have no way to correlate “the alerts got worse” with “the config changed.” Pinning the config to the model artefact — so that promoting a model promotes its tuning as an atomic unit — is the single most valuable habit in this discipline. This is an observed pattern across the operational-anomaly engagements we work on, not a benchmarked figure; the mechanism, though, is not subtle. How rollback actually works when a retune goes bad Here is the payoff, and it is concrete. Suppose an anomaly model on an energy asset is retrained on a rolling monthly window and its sensitivity retuned to catch a class of slow-drift faults. Two weeks later the on-call rotation reports the channel is unusable. What can each kind of team do? Code-only team: They know the script is unchanged. They do not know which data window the live model was fit on, whether the threshold moved, or what the previous false-positive rate was. Their realistic options are to mute the alerts (and risk missing the real fault the retune was meant to catch) or to re-derive a working model from scratch under time pressure. Data-plus-model-plus-config team: They pull the currently-live state, pull the last-known-good state, and diff them. The diff shows the threshold dropped by X and the training window now includes an anomalous fortnight that skewed the baseline. They roll back the model-plus-config atomically, confirm the false-positive rate returns to its prior level, and then investigate the drift calmly. The measurable outcomes are the ones an ops lead cares about: rollback time after a bad retune, the fraction of production alerts traceable to a specific data/model/config version, and the avoided re-tuning cost when a sensitivity change degrades behaviour at the on-call bandwidth limit. None of these are hardware benchmarks — they are operational measurements you can put on a dashboard, and they move sharply once the triple is versioned as a unit. Reproducing a past state instead of re-deriving it is the whole game. A model registry entry that resolves model@v14 to its exact weights, its data-slice hash, and its config is what lets an engineer reconstruct the model that fired an alert three weeks ago. Without that, “reproduce the alert” is not a task — it is a research project. How does version control fit an existing SCADA or observability stack? The fear ops teams voice, reasonably, is that “ML version control” means adopting an MLOps platform and adding operational overhead to a SCADA environment that is deliberately conservative. It does not have to. The versioning substrate is metadata and content-addressable storage; it sits beside your telemetry historian and observability stack rather than inside the control path. In practice this looks like: telemetry snapshots hashed and manifested with a tool like DVC or LakeFS, trained artefacts logged to an MLflow-style registry with the data-slice hash and config recorded as tags, and the promotion step that pushes a model to the inference host stamping the deployed version into the same observability system (Prometheus labels, log tags) that the on-call engineer already reads. The anomaly alert then carries the model version as a label, so the trace from “alert I saw” to “state that produced it” is one query, not an archaeology dig. This is the reproducibility layer that a production AI monitoring harness depends on — you cannot audit a tuned anomaly model you cannot reconstruct, and our validation-pack work for industrial and energy operations treats versioning as its precondition, not an add-on. None of this touches the deterministic control loop. The versioned assets are the ML inference layer’s dependencies; the SCADA system’s real-time path is unaffected. That separation is what makes the discipline adoptable in an environment that will, correctly, refuse to bolt a platform onto its control network. When is lightweight versioning enough versus a full registry? Not every industrial or energy anomaly system needs a full MLOps registry. The honest scope is the engineering layer — data, model, and config versioning — and for a single model on a single asset, that can be a disciplined directory convention plus content hashing. The decision hinges on how many models you run, how often they retune, and how much reliability audit you are subject to. Signal Lightweight discipline is enough You need a registry Number of deployed models A handful, one team Dozens across sites Retune frequency Rare, manual, reviewed Scheduled / automated retraining Audit requirement Internal only Formal reliability audits, external review Rollback expectation Minutes-to-hours acceptable Automated, near-immediate Team continuity Same engineers own it Handoffs across shifts / vendors The trap is to skip both — to have neither a lightweight discipline nor a registry, which is where the code-only team lives. Even the lightest version of this practice (hash the data slice, tag the artefact, pin the config) delivers most of the traceability benefit. The registry earns its keep when retraining is automated or audits are formal, because at that point manual discipline stops scaling and the cost of a non-reproducible model is measured in audit findings, not just a rough night on-call. For teams weighing the benchmarking side of that hardware-and-workload decision, our SPECint explainer for anomaly-detection workloads covers a related trap: reading a spec-clean number as if it predicted operational behaviour. FAQ What’s worth understanding about machine learning version control first? In practice it means versioning the data slice, the trained model artefact, and the tuning configuration together as one reproducible state — not just committing the training script to Git. A model’s behaviour depends on the data window it saw, the weights training produced, and the thresholds set afterward, none of which the code alone captures. Versioning the triple lets you reconstruct exactly the state that was live at any point. What do you actually version in an anomaly-detection deployment — code, data, model artefact, or tuning configuration? All four, resolved to a single reproducible state: the code (did the algorithm change?), the data slice (did the definition of normal shift?), the model artefact (is this the model that was live?), and the tuning configuration (did someone move the knobs?). The tuning configuration is the most commonly untracked and highest-leverage of the four, because a one-line threshold change shifts the false-positive rate directly against the on-call limit. How does versioning the sensitivity/tuning configuration alongside the model help when a retune degrades the false-positive rate? When the config is pinned to the model release, promoting a model promotes its tuning atomically, so you can correlate “alerts got worse” with a specific versioned change. Without that link, a threshold change is invisible in the code history and the alert-flood has no diagnosable cause. With it, the degradation becomes a diff you can inspect and reverse. How does ML version control let you roll back or reproduce a deployed anomaly model after a bad update? Because each deployed state resolves to its exact weights, data-slice hash, and config, you can pull the current and last-known-good states, diff them, and roll back the model-plus-config as an atomic unit. Reproducing a past model state instead of re-deriving it from scratch is what turns a crisis into a controlled fix. It also lets an engineer reconstruct the model that fired an alert weeks earlier. How does version control integrate with an existing SCADA / observability stack without adding operational overhead? The versioning substrate is metadata and content-addressable storage that sits beside your telemetry historian, not inside the real-time control path. Telemetry snapshots are hashed, artefacts are logged with data and config tags, and the deployed model version is stamped into the same observability system the on-call engineer already reads. The SCADA control loop is untouched; only the ML inference layer’s dependencies are versioned. How does traceable versioning support reliability audits and reproducing an alert an on-call engineer saw weeks ago? An audit is only trustworthy if the versioned state it references can be reconstructed, so traceable versioning is the enabling practice for reliability review. When each alert carries its model version as a label, tracing from “alert I saw” back to “the exact model, data, and config that produced it” is a single query. That is the reproducibility substrate reliability audits and post-incident reviews depend on. When is a lightweight versioning discipline enough versus a full MLOps registry for an industrial or energy anomaly system? Lightweight discipline — hashing the data slice, tagging the artefact, pinning the config — is enough for a handful of manually-retuned models under internal review, and it captures most of the traceability benefit. A full registry earns its keep once retraining is automated, models number in the dozens across sites, or formal audits demand near-immediate, automated rollback. The failure mode is skipping both, which is exactly where the code-only team sits. The uncomfortable question to sit with is not “do we have version control” — everyone has Git. It is narrower and harder: can you name the exact data window, weights, and threshold that produced the last alert your on-call engineer acted on? If reconstructing that state is a research project rather than a query, the reproducibility gap is already open, and it is the same gap that makes a reliability audit of a tuned anomaly model impossible to close.
