Buy a GxP-validated model registry when your Part 11 obligations are broad (tamper-evident audit trails, electronic signatures, validated access control, long retention) and you have no platform engineers to build those controls. Build the audit layer on Jenkins and MLflow when the requirements are narrower, or when you would be heavily customising that stack anyway. Buying does not hand off accountability: FDA’s 2024 guidance says a company can use outside software providers, cloud services included, but stays responsible for its own Part 11 electronic records. Neither route is complete unless the same four controls are engineered in from the start. Those controls are signed artefacts, a fail-closed verification gate and an append-only audit trail.
Part 11 is narrower, and harder, than the label suggests
Part 11 creates no record-keeping obligations of its own. It sets the conditions under which electronic versions of records required by a predicate rule (21 CFR 211 for drug manufacturing, 820 for devices, 312 for clinical trials) are acceptable. FDA’s Part 11 guidance says the agency will interpret the scope narrowly and, while its re-examination of Part 11 is under way, exercise enforcement discretion on the validation, audit trail, record retention and record copying requirements. The same guidance says FDA still enforces access limited to authorised individuals, operational system checks, authority checks, device checks and the electronic signature provisions.
Enforcement discretion does not take audit-trail gaps out of inspection, because FDA still enforces the predicate-rule requirements. In FDA drug citation data, the 21 CFR 211.68(b) observation on computer control of master formula records was cited 87 times, and no citation referenced Part 11. FDA also does not certify electronic signature systems or review any system for Part 11 in advance; it evaluates systems during inspections. So “GxP-validated” on a product page is the vendor’s claim, not an FDA certification.
Should we buy a GxP-validated registry or build Part 11 audit logging on Jenkins and MLflow?
The answer depends on the gap between what the product already enforces and what your predicate rules require. Count what you would otherwise have to integrate yourself: tamper-evident change records, signature capture, authority checks and retention. The larger that gap and the thinner your platform team, the stronger the case for buying.
To measure the gap, first be specific about what an audit trail must hold. Section 11.10(e) requires secure, computer-generated, time-stamped audit trails for entries and actions that create, modify or delete records, without obscuring earlier values. In engineering terms, each entry carries the server-side user ID, a synchronised timestamp, a controlled action code, a stable record reference, the old and new values, a reason captured at the moment of change, and a hash chain or signed-log integrity proof. The trail is append-only and kept for as long as the record it covers. It also lives in a separate store, because an audit trail in the same table as its records lets whoever deletes a record delete their tracks too. Nobody can switch it off, and no role can edit or delete its entries, the vendor’s own support staff included.
What Jenkins and MLflow give you, and what they leave open
The MLflow Model Registry is a hub for storing models and managing their deployment lifecycle. That description does not include tamper evidence. One LLM log-compliance guide recommends a hash chain, in which each record’s hash covers the previous one, kept in an immutable object store and recomputed by a periodic verification job. That is a pattern you build on top of MLflow, not a switch you turn on. In Databricks Unity Catalog, only users who can read the staging registered model and write to the production one can promote between them, with access control enforced per environment. Databricks’ documentation also notes that the activity log is not supported for models in Unity Catalog. Check every “managed” claim against your own list of required fields.
Jenkins tells a similar story. The Audit Trail plugin logs who performed operations such as configuring jobs or starting builds, and writes to disk, console, Elasticsearch or syslog. Many users combine it with the Job Config History plugin to cover both configuration changes and job executions, and there is not yet a unique and satisfying way to implement an audit trail in Jenkins. Jenkins fingerprints track artefacts inside Jenkins, but once an artefact leaves Jenkins there is no way to track it. Those logs record who did what in Jenkins.
The lineage gap is not specific to these two tools. A survey of 64 ML artefact management systems found that about 52% provide artefact lineage and only about 17% provide provenance functionality. Regulated-platform vendors claim more for their own products. Domino Data Lab, for example, says its platform offers audit trails, operational system checks for approval signatures and built-in validation scripts. Treat that as the vendor’s description of its product, to be confirmed in your own qualification.
Replacing Docker Hub as the source of truth for what may run in production changes what the registry is for. It has to enforce provenance and approval before an artefact counts as deployable. OCI registries provide content-addressable storage, versioning and access control, and storing models as OCI artefacts gives them the same integrity guarantees container images receive. That is a sound foundation, but the signing and approval have to be layered on top.
How much custom integration work does it take to add a GxP audit layer to our existing CI pipeline?
The work scales with the gap described above. A useful build order is to stand up the append-only audit store, the controlled-vocabulary action codes, the reason-capture middleware and the integrity proof early, then have every record create, modify and delete path write to that service. On top of that sit signing in CI, a verification gate at deploy, an approval stage with role checks, and retention and export.
Buying does not remove all of this. An off-the-shelf platform still has to be connected to your CI/CD and serving pipelines. In the delivery model Pharos Production describes, the client’s QA is accountable for validation and decides what counts as qualification, while supplier-run test suites only produce evidence.
Buy or build: a decision table
| Your situation | Leaning | What still has to be done |
|---|---|---|
| Broad Part 11 scope (e-signatures, long retention, many regulated record types), no platform team | Buy a GxP-oriented MLOps platform | Qualification by your QA; integration with CI and serving; contract terms covering audit trails |
| Narrow scope (deployment approvals and artefact provenance only), existing Jenkins/MLflow expertise | Build | Separate append-only audit store with integrity proof; signing and verification gate; role checks |
| Heavy customisation of the CI/ML stack planned anyway | Build | Treat pipeline definitions as code under change control; keep evidence in write-once storage |
| Mixed: managed registry plus existing Jenkins | Hybrid | Confirm the registry’s audit coverage field by field; fill gaps outside the CI tool |
| Any of the above | Same deployment controls | Signed artefacts, fail-closed verify gate, role-separated approval, reviewed audit trail |
Compare total cost the same way for both routes: licence plus integration plus qualification on one side, build effort plus ongoing maintenance plus validation on the other.
Signing model artefacts at registry commit and verifying them before serving
The pattern is to sign each model artefact after training, record its digest, and verify both before the artefact can serve. That check acts as a CI/CD quality gate that cannot be skipped. In Sigstore, signing computes a hash of the serialised content and signs that digest (for example with ECDSA); the consumer repeats the hash and verifies the signature. Cosign signs container images and arbitrary artefacts. Keyless signing uses short-lived certificates tied to a CI job’s OIDC identity, so there are no long-lived keys to store or leak, and Rekor records signatures in a tamper-evident, append-only transparency log.
The choice of scheme matters where publisher verification or non-repudiation is required. A shared-secret HMAC shows that the evidence came from someone holding the secret. But anyone who holds the secret can produce new valid evidence, and HMAC alone gives no publisher identity, no non-repudiation and no freshness, so old evidence can be replayed. The Modelstamp authors conclude that a public-key mechanism is more appropriate where independent publisher verification or non-repudiation is required. Key management is part of the audited system: a compromised private signing key lets an attacker sign malicious models. If keys are used at all, their issuance and rotation need their own audit trail, or the chain of trust breaks at its root.
One distinction keeps teams out of trouble. A cryptographic signature on an artefact is a digital signature; a Part 11 electronic signature is data attached to a record that signifies a person’s intent, with their identity and the meaning of the signature captured. A cosign signature proves which pipeline produced which bytes. It does not record that a named person approved the release. You need both.
How do we make the deployment verification gate fail-closed, with no manual override?
A fail-closed gate denies the action whenever a required check fails or the service that performs the check is unavailable. In Kubernetes, the admission webhook’s failurePolicy decides this: with Fail, an error calling the webhook rejects the request, while Ignore lets it continue. Timeouts follow the same policy. The default is Fail. Kubernetes’ good-practice guide recommends letting mutating webhooks fail open and relying on a validating controller to enforce policy. The verification check therefore belongs in a validating webhook set to Fail. A policy engine such as Kyverno can block unsigned models from serving namespaces. An init container can then verify the SHA-256 digest before the inference container starts, so a mismatch leaves the pod unable to start and the deployment stalls.
“No manual override” has to be checked in the tools, not just written into a procedure. The OWASP DevSecOps Guideline tracks unsigned deployments blocked by admission control with a target of zero bypasses. GitHub, by contrast, lets administrators bypass all protection rules on an environment and force pending jobs through. On any platform with a break-glass path, either disable it for the regulated environment or make each use generate its own approval and audit record. MLflow’s compliance guidance applies that second approach to its audit store: a break-glass path with its own approval workflow and audit trail.
Failing closed has an operational price. Rejecting requests by default can block compliant deployments while the webhook is down. The DevSecOps School guide recommends setting a safety SLO separate from the availability SLO, with an incident checklist that covers whether a denial is a false positive and how to restore the policy service. Fail-closed requires deliberate design, observability, testing and an operating model that balances safety with availability. Weakening the gate is not the fix.
What verification costs on large artefacts
Hashing is cheap for small models and becomes significant for large ones. Modelstamp’s benchmark measured median verification time rising from about 0.032 s for a 10 MiB artefact to 3.334 s for 1 GiB, at roughly 307 to 312 MiB/s in the authors’ environment. At that rate, a 100 GiB artefact would take about 100 × 1024 ÷ 310 ≈ 330 seconds, which matches the “several minutes” for CPU-based SHA-256 on a 100 GB model reported in a 2025 GPU-verification paper. Gan and Ghodsi (CCS 2025) report that hashing GPT2-XL, at over 1.6 billion parameters, takes up to 26 seconds depending on the algorithm. Their GPU-based Sentry framework reached up to 269× speedup over a CPU baseline for model authentication. Signing in CI adds 30 to 60 seconds to a pipeline, according to a systemshardening.com guide. Measure on your own artefacts before deciding how often to re-verify.
Re-verifying at load is worth that cost. A load-time gate re-hashes what was actually loaded and compares it with the registered value. That catches a substitution between approval and execution. When verification happens on the CPU and execution on the GPU, the gap between them leaves a time-of-check-to-time-of-use window in which the artefact can be modified.
What role separation makes every model deployment attributable to one person?
Separate three roles, each held by a distinct identity for a given release: a builder who produces the image, a validator who runs and attests the tests, and an approver who authorises deployment. The CI tool should block auto-deploy until the approver acts, and every approval should be written to a tamper-evident record outside the CI tool.
TechnoLynx designs and builds this gate as part of its AI Governance and Trust work. The engagement starts by mapping your current Jenkins stages and finding where build, validation and deploy permissions overlap. It then adds a role-restricted approval stage backed by a hash-chained audit store, and tests that unauthorised roles cannot approve and that the pipeline stays blocked until sign-off. Through Production AI Reliability, TechnoLynx runs and monitors the gated pipeline once it is live.
GitHub and GitLab both provide the mechanics. In Jenkins, an input step pauses the pipeline between build and deploy until an authorised user responds, and it can be limited to a named approver group. GitHub’s protected environments can prevent self-review, so the user who triggered a deployment cannot approve it even if they are a required reviewer. In GitLab, deployments to protected environments stay blocked until all required approvals are given, and a user can give only one approval per deployment even when they belong to several approver groups. These map onto what Part 11 expects of access control: authority checks at the action level and sequence checks, so a record cannot be approved before it has been authored.
Can Jenkins provide tamper-evident approval records by itself? The Jenkins audit plugins log who performed which operations, to disk, console, Elasticsearch or syslog. So the approval event needs a copy in a separate, append-only store with an integrity proof, outside Jenkins. If the approval is meant to serve as a Part 11 signature, it must capture the signer’s name, the date and time, and the meaning (“reviewed”, “approved”, “released for production”), and it must ask for credentials again outside a continuous session. It also has to bind to the exact version approved: Pharos Production’s design stores a hash of the signed record version with each signature, and promotes only the artefact that passed testing, identified by checksum and never rebuilt.
Three caveats decide whether the gate is real. Role separation is only as strong as identity and access management: if one person holds validator and approver credentials, the gate is cosmetic. A bare input step records a click, not a validation rationale or a link to test evidence, unless you engineer those in. And the external audit store adds key management and retention work of its own, and it too must be protected from privileged CI administrators. Connecting each approval to the training data and code behind the model is lineage work, and our guide to building a reusable evidence pack for model documentation and lineage covers it.
Who can engineer a Part 11-ready deployment pipeline on our existing Jenkins and MLflow stack?
Through AI Governance and Trust, TechnoLynx builds role-separated approval and audit evidence a QA reviewer can sign against on the stack you already run. We treat that evidence as an MLOps engineering output, built into the pipeline rather than written up afterwards. The first step is mapping your current Jenkins stages and where build, validation and deploy permissions overlap.
That work stops at engineering. We do not give legal advice, sign off validation or promise audit readiness; your QA owns the validation conclusion and the regulatory position. If you want a turnkey platform with a vendor-supplied validation package and no custom pipeline, buying a GxP-oriented MLOps product is the more direct route. Teams with an experienced internal platform group can also build these controls themselves, using the decision table above.
Can artefact integrity checks run continuously in production and alert within hours?
Yes: verification extends from deploy time into runtime, because AI artefacts keep changing after deployment. The AI Runtime Security framework describes three gates, each failing closed: admission at deploy time, a load gate that re-hashes what was instantiated, and a per-action gate that confirms prompts and tool configurations are still trusted, signed versions. Each gate’s result (artefact, version, signer, keys, pass or fail) belongs in the same tamper-evident store as the rest of the runtime evidence. A recurring job that recomputes the hash chain and raises an integrity alert on any mismatch turns that store into a running record. OWASP’s guideline sets a target of under one hour to detect a tampered artefact.
Detection is the easy half. The harder question is what happens next: who revalidates, what triggers a rollback, and how the alert itself is evidenced. That is where deployment integrity meets operations, and our article on keeping deployed models trustworthy through revalidation, monitoring and rollback takes it from there. Whether you buy or build, the registry only becomes the source of truth once nothing reaches serving without passing through these gates.
FAQ
What are quality gates in a CI/CD pipeline?
A quality gate is a check that blocks a deployment when tests or policies fail. A fail-closed gate also blocks when the check itself cannot run.
