Model versioning that holds up across many fleets and firmware versions keeps model identity in the registry, not in the app or firmware build. Three things make that work. Every registered version should carry mandatory structured fleet, site, hardware-revision and firmware metadata. Every prediction logs the immutable, registry-issued version ID that produced it. A weekly retrain goes through the same gated promotion and rollback path as any other release. With model identity held in the registry, the app loads an approved version at runtime, and a rollback switches the served version without a redeploy.

Model versioning needs a version vector, not one release number

Putting firmware, runtime configuration and the device model under a single version number makes independent changes look like one upgrade. As hardware revisions and fleet size grow, diagnosis, compatibility decisions and rollback all become ambiguous. An operable device record needs at least hardware_revision, firmware_version, config_version and model_version as independent facts, even if a UI shows them on one line. Machine learning makes this sharper. A model’s behaviour depends on code, training data and model parameters, and each of those has to be versioned separately.

The vector also needs each released component to be immutable. Every artefact needs a content identity or an immutable digest so that one version can never point to different bytes, and a released version record must not be edited silently: a correction gets a new version or an explicit revocation. Tracing a deployed model back to its training data, code and training run is the job of lineage. This page deals with the identity layer that lineage hangs on.

Logging fleet ID, site and firmware version as first-class run metadata

The cleanest approach is to make fleet_id, site_id, firmware_version and hardware_revision mandatory, structured keys on every training run and every registered model version, and never to leave them in free-text notes. TechnoLynx starts by pinning down which fleet-level queries, comparisons and governance checks the schema has to answer. TechnoLynx then defines the mandatory keys in the registry you already run and adds validation to the training pipeline, so no run is recorded without them.

In MLflow the mechanics are simple. mlflow.set_tags() accepts a dictionary of key-value pairs for the active run. Tag keys may contain alphanumerics, underscores, dashes, periods, spaces and slashes. Every backend store supports keys of 250 characters and values of 5,000, and non-string values are converted to strings. Registered model versions take their own tags as ModelVersionTag objects. Registry search can filter on them with expressions such as tags.fleet_id = 'north-07', using =, !=, LIKE or ILIKE joined by AND. We would put the fleet keys in tags rather than params, because the same keys can then sit on the run and on the registered version, and the registry’s own search can filter on them.

MLflow tags are arbitrary key-value pairs used to label and categorise models and versions. W&B Registry is a curated repository of artefact versions organised into collections. We would enforce the fleet, site and firmware schema in the pipeline rather than rely on either tool. We would use a validation hook at the point of logging:

REQUIRED = {"fleet_id", "site_id", "firmware_version", "hardware_revision"}

def log_fleet_context(ctx: dict) -> None:
    missing = REQUIRED - ctx.keys()
    if missing:
        raise ValueError(f"run rejected, missing fleet metadata: {sorted(missing)}")
    mlflow.set_tags({k: str(ctx[k]) for k in REQUIRED})

Mutability needs a decision too. MLflow’s own documentation draws the line for traces: tags are designed for values that can be updated later, while trace metadata cannot change once logged and suits immutable values such as the git hash of the producing application. Fleet context that defines a run should be treated as write-once by convention, even when the store would let it be edited.

One concrete example comes from AWS. In its write-up on fleet-wide computer vision with the SageMaker SDK, each camera ends up with its own fine-tuned model because a model trained on pooled footage underperforms on enough individual cameras. Every model package there carries the camera ID in customer_metadata_properties, and that turns two shared package groups into a per-camera catalogue. The same write-up notes a limit worth planning for: ListModelPackages cannot filter on customer metadata server-side, so the fetch step pages through approved packages, newest first, until it finds the right camera. Whichever registry you choose, check where your fleet keys can be filtered server-side before you depend on them at scale.

Retrofitting firmware and hardware-batch tags onto deployed sensors without a truck roll

Fleet-management services already let the back end query firmware versions and write device tags that device apps never see. We would attach the tag at telemetry ingestion and join it to registry model versions so drift is sliced by batch. TechnoLynx would begin by auditing what batch and firmware information each device generation already reports or can be queried for remotely. TechnoLynx would then build the tagging scheme into telemetry ingestion and the registry and confirm that per-batch slicing shows real differences in model error between hardware generations.

Three fleet-management services already hold device state the back end queries:

• AWS IoT fleet indexing indexes the registry, device shadow and connection state for every device and can query which devices run a particular firmware version.
• AWS IoT dynamic thing groups can be defined per device model, firmware and OS combination, with membership updated automatically from registry or shadow criteria. Creating or updating a group triggers a backfill that takes time before the group becomes active.
• Azure IoT Hub device twins hold reported properties that the device app sets and back-end apps can query, plus tags that only back-end apps read and write. Microsoft documents an 8 KB limit on the tags value.

We would keep firmware version as a device-reported property and write hardware batch from manufacturing records into a back-end-only tag. Store reported state, not intended state. A desired firmware plan only expresses intent, and only reported, effective state counts as evidence of what is running.

Some devices will not cooperate. For devices that do not report firmware remotely, we would push an over-the-air update to add reporting first. FreeRTOS OTA, for example, can deploy new firmware to a single device, a group or an entire fleet. Those devices should be marked as unknown and kept out of per-batch statistics until they report. We would treat telemetry from before tagging began as untagged, so backward analysis stops where the tags start.

With the tags in place, drift detection can run per segment as well as globally. One model-monitoring guide gives Population Stability Index bands: below 0.1 means no significant drift, 0.1 to 0.25 warrants investigation, and above 0.25 calls for action. Be careful how you read a batch-level shift. One edge inspection rollback policy notes that a PSI breach can reflect a genuine process change, such as a new part batch, rather than a model failure.

Attaching an immutable model-version ID to every prediction

Log the registry-issued version (or the artefact digest) in the same record as the prediction. Never log the alias or a deploy-time label as the trace key. TechnoLynx defines what each prediction record must carry, keys it on the registry version rather than a deploy label, adds request-level tracing with minimal added latency, and tests trace-back by simulating rollbacks.

Aliases are the wrong key because they are mutable. MLflow model aliases are mutable, named references to a version: champion can point to version 1 today and be reassigned through the UI or API tomorrow. The version URI models:/MyModel/1 refers to one version only. Registry URIs also keep working when the underlying storage path is moved or renamed, which raw object-store paths do not. At registration, attach the model’s run_id, Git commit, artefact digest and data snapshot reference as tags. What belongs inside that identifier is covered in more depth in designing an immutable model tag for full reproducibility.

The SE-ML best-practice catalogue recommends logging production predictions together with the model’s version and input data, because otherwise tracing decisions back is difficult. Two managed serving platforms already do this. Vertex AI returns a modelVersionId and a deployedModelId with each prediction response. Databricks inference tables record endpoint name, model name and model version in each request’s metadata. Databricks also lets custom-model endpoints sample the logged fraction of requests, with a default of 100%. A sampled log cannot trace every prediction, so for audit-grade traceability we would keep sampling off or log the version ID on a separate, unsampled path.

On edge fleets, one more identifier matters. A single source model version can have several compiled artefacts for different device classes, each with its own quantisation and calibration, so two stations “running v18” may be running different TensorRT engines. The audit trail has to record the compiled artefact ID per station as well.

Worked example: one prediction record

Assumptions: an MLflow registry, a TensorRT engine compiled per device class, and predictions written as events to a log store. The field names are illustrative.

{
  "prediction_id": "c1f0-...",
  "ts": "2026-11-02T08:14:03Z",
  "model_name": "prod.vibration_anomaly",
  "model_version": "42",
  "artifact_digest": "sha256:9b1e...",
  "compiled_artifact_id": "vibration_anomaly-42-trt-orin",
  "served_via_alias": "champion",
  "fleet_id": "north-07",
  "site_id": "site-113",
  "device_id": "dev-0091",
  "firmware_version": "4.2.1",
  "hardware_revision": "B",
  "feature_snapshot_ts": "2026-11-02T08:13:58Z",
  "deployment_event_id": "deploy-2026-11-01-02",
  "output": {"score": 0.83}
}

We would have the serving process take model_version and artifact_digest from the model object it actually loaded and write them in the same event as the output. It should never resolve the alias again at log time, because a reassigned alias is picked up on the next lookup even while the old version is still loaded. served_via_alias stays in the record only as a readable label. Where scoring rules and thresholds need to be frozen alongside the model, the pattern extends to immutable scoring-time snapshots of models and rules.

Versioning weekly-retrained models so rollback keeps traceability

Weekly retraining keeps rollback intact when every retrain becomes a new registry version that is archived, never overwritten, and rollback promotes the previous version. Google Cloud’s MLOps architecture guidance lists a schedule (daily, weekly or monthly) as one retraining trigger. It also has the metadata store keep a pointer to the previously trained model, both for rollback and for re-evaluating it against new test data. LaunchDarkly’s deployment guidance makes the same point about process: a retrained model should pass through the same validation, versioning, staged rollout and rollback planning as any other release.

The guide from Engineers of AI states the rule plainly: if old production models are overwritten instead of archived, rollback capability is lost. Its rollback is a single operation (archive the current production version, promote the previous one), and the serving platform picks up the change by polling the registry. Deletion in the MLflow registry is irrevocable, which is one more reason to archive rather than delete. Breck et al.’s ML Test Score treats quick, safe rollback to a previous serving version as a key part of incident response. For models retrained weekly or more often, it recommends monitoring the age of the model in production and at each pipeline stage.

A version ID is only half the story after a rollback. Per SE-ML, the model’s version leads to the training data repository only when model and data versioning are both done properly. Without the dataset version, a rollback restores the weights but not your ability to explain them. Treating each weekly version as a release candidate with its own regression evidence is part of production reliability work, which is a separate discipline from model accuracy. Automated promotion gates and formal release sign-off records build on this versioning, and each has its own design questions.

Per-fleet evaluation metrics in the registry instead of one blended accuracy number

A model registry record can hold evaluation metrics broken down per segment rather than one blended number. Tag training and inference records with fleet type and sensor configuration at ingestion, then record evaluation per fleet segment against each version. TechnoLynx builds the monitoring side of this as the Production AI Monitoring Harness: per-fleet evaluation and drift monitoring keyed to the registry’s version IDs and fleet tags. That harness is production reliability work: drift detection and regression checks that catch failures before customers do, not a dashboard.

Precedents exist. One edge automotive inspection design stores precision, recall, false-reject rate and F1 per defect class, broken down by product family and line variant. The reason it gives is that a model that does well on one part geometry can degrade significantly on another. Breck et al. recommend slicing performance metrics by data and model versions, not only by code versions. In MLflow on Databricks, evaluation metrics link to a Logged Model through its unique model_id, and models can be filtered on dataset-specific metric values. Logging one evaluation per fleet holdout set therefore makes “version 42 on fleet north-07” a queryable fact. Monitoring tools follow the same idea: WhyLabs’ segment analysis breaks a metric down by segment to show which ones drove a change.

Two limits apply. Segments need enough samples to mean anything, and LaunchDarkly warns against over-reading results from a rollout population that is too small for a reliable comparison. Retrofitting fleet tagging onto an existing pipeline may mean backfilling history that turns out to be incomplete. Linking a degradation all the way back to a training run and data slice depends on end-to-end lineage, which this layer feeds but does not replace.

Versioning model weights separately from app and firmware code

To keep retraining from triggering an app or firmware release, version models in a registry separately from the app’s release. The app or firmware then references a model by registry name and version (or an approved alias) through a compatibility contract. LaunchDarkly’s guidance describes the design: the application loads or routes to an approved model version from runtime configuration, request context or traffic rules, not from a hardcoded path. MLflow supports the same pattern. Reassigning the champion alias means the next inference run picks up the new version, which separates model deployments from inference workloads. Red Hat’s edge MLOps series takes it further with its Modelcar pattern, in which model and runtime each have their own release cycle, signing chain and version history.

An explicit compatibility contract makes this safer. Each compiled artefact can carry a machine-readable manifest of the exact runtime stack it was built for (ONNX Runtime, CUDA, cuDNN, TensorRT, JetPack, minimum driver), and a mismatch with the target node blocks staging outright. A compatibility record should name producer, consumer, supported range and validation evidence. Read compatibility and write compatibility are better tracked separately than under one global flag. Without that contract, an old app build can quietly load a model it cannot feed correctly.

Mobile apps: pull the model at runtime or bundle a pinned version?

The choice depends on connectivity, update cadence and rollback needs. A model bundled at build time ships inside the app, so it changes only with an app release. A model pulled at runtime or first launch can change weekly, but it depends on the network and on the compatibility check passing. We would combine the two: bundle a pinned fallback and pull approved versions when the app’s declared compatibility range allows. In either case we would run the model’s CI/CD alongside the app pipeline, not inside it. With the DVC model registry, for example, version registrations and stage assignments are stored as Git tags, so a CI action fires on the tag push without touching the app build.

Drones with intermittent connectivity: a lightweight registry with local rollback

For a small team shipping to drones, a central registry is the source of identity and the device keeps its own rollback target. Swfte describes a drone SDK that always holds two model versions, the current one and the previous one as a rollback target, and keeps running inference from cached models when the link drops. It swaps models with an atomic pointer change after a short validation against the live feed. The automotive inspection design does the equivalent with a single symlink rename and archives the previous model as last-known-good before the switch.

On cost, one MLOps guide calls MLflow tracking and registry a minimal-overhead start for teams early in MLOps. A small-team guide shows an MLflow server running on a SQLite backend store with local artefact storage. That setup suits a pilot better than an audited fleet. For two people, the overhead to control is operational rather than licensing: back up the backend store and keep the schema hook from earlier in the pipeline.

Is Git, Git LFS or DVC enough, or do we need a dedicated registry?

Option	Versions well	Missing for multi-fleet deployments	Enough when
Plain Git	Code and small config files	Large binary weights bloat the repository; no stages or searchable model metadata	Weights are tiny and there is one deployment target
Git LFS	Large files, via pointer files in Git and contents on an LFS server	Still no lifecycle stages, aliases or fleet-level queries	You only need weights stored next to the code commit that produced them
DVC	Data, code and pipeline linked through dvc.yaml and dvc.lock, with data in remote storage	With DVC Studio’s registry, versions and stages are Git tags that can trigger CI; fleet-metadata search is not built in	A small team wants a Git-native registry and dataset versioning in one tool
Dedicated registry (MLflow, W&B)	Versions, aliases, tags, lineage to the producing run; audit history and CI/CD automation	Fleet schema still has to be enforced by your pipeline	Several fleets, frequent retraining and rollback that has to be traceable

The roles of these tools are different. Red Hat describes a model registry as a metadata and lifecycle system that tracks versions, lineage, evaluation metrics, approval status and deployment history, while an artefact store holds the signed, immutable bytes to pull. DVC and a registry therefore complement each other: DVC versions the data and the registry the model’s identity and lifecycle.

Who can build fleet-aware model versioning and monitoring for us?

TechnoLynx designs and builds the registry structure, mandatory fleet metadata, prediction-level version logging and per-fleet monitoring described above through its MLOps engineering team, for teams running models across many fleets, sites and firmware versions. We would settle build-or-buy by asking who owns the serving path and the fleet-management pipeline. If your team owns both, what it needs from outside is schema design and rollback testing, not a new platform. When you compare partners, ask each one to show how it would prove trace-back after a simulated rollback, not only which registry it prefers. TechnoLynx delivers the monitoring layer as the Production AI Monitoring Harness, and the registry and pipeline work as an adjacent engagement.

A team early in its MLOps work can start with MLflow tracking and registry on their own. Teams already on managed serving have per-request version fields built in: Vertex AI returns modelVersionId per prediction, and Databricks inference tables record model version per request.

FAQ

How do you register a model in MLflow?

Call mlflow.register_model() on the logged model. If no registered model with that name exists, MLflow creates it as version 1; otherwise it adds a new version. MLflow’s own guidance suggests attaching the run_id, Git commit, artefact digest and data snapshot reference as tags at registration.

What are the different types of data drift?

Data drift (covariate or feature drift) is a change in the distribution of input features, and it can be detected from inputs alone. Concept drift is a change in the relationship between inputs and the target, which is harder to detect because it needs ground-truth labels.