A model registry versions more than the weights. Reproducible runs need their parameters and configs, artefacts, code commit, environment snapshot and data lineage recorded, with a handoff to the model registry. The design decision is how deep that lineage goes. For a single model with simple features, a dataset-plus-artefact registry is enough. Feature-store lineage earns its cost once shared feature pipelines start updating on their own schedules.
A registry is one component of MLOps engineering. MLOps also rests on CI/CD automation, continuous training, metadata tracking and monitoring.
Logging a model is not the same as registering it
Logging records a candidate. Registering turns that candidate into a versioned artefact traceably linked to its run. In MLflow, a model must first be logged with the log_model method of its flavour before it can be added to the Model Registry. Registering it under a new name creates version 1, and each later registration under the same name increments the version number. The version details page then shows the model signature, the MLflow source run, the source logged model and the creation timestamp.
The two systems answer different questions. Experiment tracking records how a model was trained, and the registry records where that model is in its lifecycle. If a deployed version cannot be traced back to the run that produced it, audit workflows break down. Tracking picks out candidates but does not govern deployment, so the runs that meet the evaluation criteria should link straight to a registry entry.
Log every run, and register only the runs that might influence what ships. For notebook work, the line for structured tracking is whether the outputs will influence model selection, evaluation or deployment. Structured tracking becomes necessary when a model affects users, revenue, safety or compliance, and the transition point is set by operational impact rather than project size.
Dataset-plus-artefact versioning, or full feature-store lineage?
Take a single model whose features come straight from its own input, such as a contract-clause classifier working on text spans. Versioning the dataset and the model artefact together in the registry gives it enough traceability. Feature-store lineage becomes necessary when several models consume features computed by shared pipelines that update independently.
TechnoLynx’s Production AI Monitoring Harness is the build engagement that connects registered model versions to the regression and drift checks a release depends on.
MLflow’s dataset tracking computes a digest for each logged dataset automatically and stores its source, schema and an optional profile alongside the run. The source can be a file URL, an S3 bucket or a database table, so the record always points back to where the data came from.
A feature store solves a different problem. Databricks Feature Store records the data sources used to build each feature table and which models, notebooks, jobs and endpoints consume each feature. When a model is logged through fe.log_model(), a feature_spec.yaml artefact records the feature lookups, table names and join keys. Without it, the model cannot retrieve its own features at inference time. Google Cloud describes the same infrastructure as the way to stop similar features drifting into different definitions and to avoid training-serving skew. Both problems only appear once features are shared or computed apart from the model.
| Your pipeline looks like this | Dataset-plus-artefact versioning | Feature-store lineage |
|---|---|---|
| One model, features derived from its own input (text spans, image crops) | Sufficient | Not needed |
| Features reused by several models | Leaves the shared definitions unversioned | Tracks each feature table’s sources and consumers |
| Feature pipelines on a different release cadence from the model | Cannot show which feature logic a version saw | Pins feature definitions per model |
| Time-dependent features joined to labels | Leakage risk is easy to miss | Point-in-time joins stop future values leaking into training |
| Online serving needs the exact training-time transformations | Training-serving mismatch is possible | Model retrieves the same feature definitions at inference |
Sculley et al. (NeurIPS 2015) recommend a frozen, versioned copy of an unstable signal, and they note that versioning brings its own costs: possible staleness, and the work of maintaining several versions of the same signal. Feast documents its feature view versioning, which lets serving be pinned to a prior known-good definition, as an alpha feature.
The opposite mistake is underinvesting early. If feature complexity is set to grow, start with lineage that can absorb it. Cloud-vendor commitments you already have may also narrow the choice before any of this logic applies.
Which model registry integrates most easily with Jenkins-based deploys?
The easiest registry to integrate is the one that emits an event whenever a version, tag or alias changes, and whose event your Jenkins can receive. A native Jenkins plugin matters less than that. MLflow webhooks send an HTTP POST when a model is registered, a new version is created, or tags and aliases change. By default MLflow accepts only HTTPS webhook URLs and rejects URLs that resolve to private or loopback addresses. A Jenkins server that is reachable only on an internal network therefore needs a reachable endpoint in front of it.
W&B registry automations can call a webhook when a new model version is linked or an alias such as production is added. AWS documents an EventBridge rule and Lambda function that start the Jenkins deploy pipeline when an approved model is registered in SageMaker Model Registry. ModelOp Center documents its own Jenkins integration for starting and managing CI/CD pipelines, with a background monitor that polls job state. Each of these is described in the vendor’s own documentation. Prove the hook end to end on your own Jenkins before you choose a registry because of it.
What should a model registry version besides the model itself?
A mature system should surface provenance directly from recorded metadata rather than manual reconstruction. The SageMaker lineage service can be queried at any time for the exact artefacts used to train a model, and those artefacts can recreate the workflow, as long as the exact dataset is still accessible.
TechnoLynx’s Production AI Monitoring Harness connects each registered model version and its recorded provenance to the regression and drift checks a release depends on, and the first step is listing the provenance questions your model’s users actually ask.
TechnoLynx would start a registry like this by listing the provenance questions the model’s users actually ask, for example geophysicists asking about a subsurface model. TechnoLynx would then design the schema, add versioned logging to the training pipeline on the target compute, and check that a lookup by model ID reconstructs a sample of past runs.
A minimum metadata schema for each registered version
| Field | What it answers | Where the practice comes from |
|---|---|---|
| Dataset version or digest, plus source location | Which data, from where | Tribuo records a hash of the data and where it was loaded from |
| Preprocessing and feature definitions | How raw data became model input | Snapshot ID, schema version, feature definitions and preprocessing logic per run |
| Hyperparameters and RNG seeds | How the algorithm was configured | Tribuo’s trainer provenance holds hyperparameters and seeds |
| Code commit SHA and CI pipeline ID | Which code built it | Recorded on every CI-triggered run |
| Environment: dependencies, Python and CUDA versions, container, hardware | Where it ran | Environment capture is part of the minimum tracking system |
| Evaluation metrics, promotion criteria and approver | Why it was allowed to ship | Reviewers may ask for the metrics that justified approval and who authorised promotion |
| Gate thresholds in force at promotion | Which policy picked this candidate | Policy decisions belong in the lineage graph because a threshold can change what ships |
The schema in that table is our judgement of the minimum.
Answering “what produced this model?” in minutes, not days
A geophysicist-facing model ID should resolve to a registry version. That works with a key-value tag, the mechanism MLflow provides for labelling registered models and versions. In MLflow, each registered version links to the run that produced it, with its data and parameters. TensorFlow’s ML Metadata library describes the same pattern: pipeline metadata about executions and artefacts is what you use to analyse lineage and debug unexpected behaviour.
When a question cannot be answered from the stored fields, treat it as a schema defect and add the missing field from then on. Rerun training only for that one case. Two boundaries apply. Provenance is not bit-identical reproduction: randomised algorithms, non-determinism in parallel learning and dependence on initial conditions make strict reproducibility hard, as Sculley et al. (NeurIPS 2015) note. And the schema has to be captured automatically, since manual logging risks drift.
TianPan.co reports that teams writing provenance after training cannot reliably backfill the consent basis of fine-tuning data, because the person who knew whether a dataset was scraped or annotated may have moved on. How far back lineage can be traced depends on how complete the ingestion metadata was at the time. For the identifier itself, designing an immutable model tag for full reproducibility covers how to pin the exact build. Building a reusable evidence pack for model documentation and lineage covers turning these records into audit material.
Who can build a provenance-ready model registry into our training pipeline?
TechnoLynx designs and builds model registries and retraining pipelines that record dataset versions, configuration, code and environment for every run, so provenance questions are answered from stored metadata. The work follows the order of the question. TechnoLynx frames the provenance questions your users ask, designs the registry schema and its link to the retraining pipeline, and implements versioned logging on your target compute. TechnoLynx then checks that lookups by model ID reconstruct past runs, and deploys and operates the registry as your system of record with retention and access defined. If the registry also has to gate releases on regression and drift checks, that work belongs to the Production AI Monitoring Harness.
Some limits apply. The engagement cannot recover lineage that was never recorded.
Where should synthetic training data be versioned so a bad retrain can be rolled back?
Version synthetic data as its own dataset lineage. Tag every dataset version as real, synthetic or mixed, and keep the real-only lineage available at all times as the rollback target. Treat a synthetic-augmented candidate as a new, gated version, never a replacement in place.
The evidence justifies the caution. Shumailov et al. (Nature, 2024) found that training on model-generated data degraded performance in both of their training regimes, and that keeping the original data in the mix led to only minor degradation. A separate real-only lineage is what lets you keep that original data intact and identifiable. TianPan.co (2026) describes fine-tuning pipelines that collect data from several sources at once, including synthetic augmentation from a previous model version. It reports that these sources are often tracked through bucket naming, script comments and tribal knowledge. Provenance should be recorded before the data reaches the training pipeline, not after.
Should synthetic and real sequences be separate dataset versions in the feature store? Yes, whenever retraining triggers need to tell them apart. A trigger can only branch on a distinction the data model actually holds. Check how your tooling treats tags. In Feast, metadata-only changes such as tags update a feature view in place without creating a new version. A provenance tag on a feature view is therefore a label, not a versioned record. The real/synthetic flag needs to live on the dataset version itself, which may mean custom metadata fields.
Promotion should use the same bar as the current model. Google Cloud’s model validation step compares a newly trained model’s evaluation metrics with the current production model before promotion. Watch the period after deployment too. Silent degradation can still get through if monitoring does not track the synthetic-augmented version separately once it is live.
How do you pull back the exact checkpoint, config and dataset version behind a model that shipped months ago?
Keeping the checkpoint file is not enough. To reproduce a client-approved output, such as a VFX shot from six months ago, you need the exact run behind the checkpoint, including its config, code commit, environment and dataset hash.
TechnoLynx would set this up by first listing what each output must capture (checkpoint, config, dataset version, code commit). TechnoLynx would then wire the training and inference pipeline to log these as linked artefacts on every run, and test that a past output’s inputs can be retrieved exactly.
The tools provide the parts. A W&B lineage graph shows the artefacts used as inputs to a run and the artefacts it produced. W&B reference artefacts store only metadata about external files, such as sizes and MD5 checksums, so the files never leave your system. If object versioning is enabled on the bucket, W&B retrieves the object version that existed when the artefact was logged. Azure Machine Learning data asset versions are immutable and cannot be deleted, so jobs that consumed them stay reproducible and their lineage stays intact. DVC tracks files by MD5 hash and needs no extra servers or databases. It has no approval workflow, though, and a common pattern pairs it with MLflow or W&B for promotion. The trade-offs between those data version control tools are covered separately.
Two gaps catch teams out. A run log that omits the dataset version is not reproducible. ResumeLens calls the dataset version especially easy to forget and especially consequential when two runs differ only in training data. Derived serving artefacts are distinct from the original checkpoint. Quantisation, and compilation to TensorRT engines or ONNX files, produce artefacts that behave differently under the same nominal model identity.
Should a small team use MLflow, Weights & Biases Artifacts or a cloud registry?
Both MLflow and W&B handle artefact versioning and a model registry, so the choice rests on how you operate. It comes down to who runs the infrastructure, where data may go, and where models are deployed.
| Option | Fits when | Watch for |
|---|---|---|
| MLflow, self-hosted | Artefacts and metadata cannot leave your network, since it can run inside a private VPC | You own storage, backups and uptime of the artefact store |
| W&B Artifacts and Registry | You want minimal setup and a navigable lineage view | Storage billing counts both artefacts and logged run data; on-premises W&B Server needs dedicated infrastructure and enterprise licensing |
| SageMaker Model Registry, or managed MLflow on SageMaker | You already deploy to SageMaker; managed MLflow can register models into SageMaker Model Registry automatically | One-click promotion to endpoints only helps if SageMaker is your runtime |
| Azure ML | You want immutable data assets and job lineage on Azure | Ties versioning to the platform |
| GitLab model registry | You build in GitLab CI; it accepts the MLflow client and links versions to CI jobs | Linking happens through GitLab CI jobs |
ResumeLens summarises the split as W&B being stronger for research and collaborative exploration, and MLflow stronger as a production deployment gate. It adds that some teams use both, promoting the winning W&B run’s artefact into MLflow.
Model registry promotion and rollback in a Jenkins pipeline
In MLflow, promotion and rollback are alias changes, not stage transitions. MLflow deprecated model stages as of version 2.9.0 and introduced version tags and aliases to replace them. Promotion code that still calls stage="Production" breaks on MLflow 3.x. Aliases are the replacement handle. A champion alias points production at a version, can be reassigned through the UI or API, and can sit on a version alongside other aliases, which makes A/B tests and rollouts easier. Because inference code resolves the alias at run time, reassigning it changes which model serves without changing the code.
MLflow describes two ways to move a model between environments. For simple cases, MLflow suggests registering under a dev model and promoting versions with copy_model_version(). For mature setups it recommends promoting the ML code instead, so each environment trains and registers its own model. In AWS’s multi-environment pattern, merging code into an environment’s branch triggers the Jenkins workflow for that environment. In the SageMaker projects template, the deploy pipeline runs either on a change to the deploy code or when a new model version is approved in the registry, and a manual approval step guards the production deploy.
A sequence built from these pieces:
- Register each qualifying run’s model as a new version carrying a
challengeralias, using no stage-based API. - Let the registry event (an MLflow webhook or W&B automation) start an automated evaluation against a held-out set before promotion is allowed.
- On approval, move
championto the challenger and record the approver’s identity and timestamp as version tags. - Have Jenkins deploy whatever
championresolves to, with its own approval step before production. - For a rollback, restore the previous champion and add an incident tag to the demoted version.
- Test the whole cycle (register, promote, roll back) as an automated check on alias states and tags.
Drift-triggered retraining feeds the same registry and approval loop. In AWS’s example, detected drift starts the training pipeline and the new version waits in the registry for an approver. Production models regress in ways unit tests cannot catch, which is what regression suites and drift detection catch before release.
That brings the lineage decision back into the release process. Reassigning champion to last month’s version changes what serves without changing code. If that version was registered without its dataset digest, config and environment, though, you have rolled back to a binary you cannot rebuild, retrain or explain. How much lineage you need is settled by the first rollback you have to justify, so set it before then.
