Yes, but only partly. If native model tracking failed, you can still rebuild part of a deployed model’s lineage by correlating CI/CD build records, deployment timestamps and inference-service logs. The result is a partial reconstruction with built-in timing and correlation errors. It is not evidence-grade proof that a specific binary produced a specific decision, so it should never go to an auditor presented as complete lineage.

An arXiv paper on audit trails for large language models uses a bank as its example. To investigate, the bank has to establish which fine-tuned model version was serving at the time, which configuration was active and what approvals governed the deployment. In many deployments that information is scattered across experiment trackers, CI logs, configuration files and email threads. The order of work follows from this: list the records that survive, fix the clock error margin, join build and deploy events to inference timestamps, write down the gaps, and stop where the evidence stops.

Which build was serving when the experiment tracker stopped recording?

Find the deploy events on either side of the window, tie each one to the CI build that produced its artefact, then assign each inference request to whichever deployment was live at its timestamp, with the window widened by the error margin. The result is a partial reconstruction whose gaps and error margins must be documented. It does not replace a tracked record.

TechnoLynx scopes this kind of reconstruction under Production AI Reliability. The engagement starts with an inventory of which serving windows have no tracked lineage and which build, deployment and inference records still exist for each one.

Each of the three streams has its own timing trap.

The CI build record. Jenkins records two different times for each build. getTimestamp is when the build was scheduled, and getStartTimeInMillis is when it started running on an executor. The Jenkins API documentation gives its own example: a build scheduled at 1pm waits an hour for an idle agent and moves to the building state at 2pm. For lineage, the executor start time plus the recorded duration (stored in milliseconds) gives the interval in which the build actually ran. The scheduled time only tells you when someone or something asked for the build. Triggers do not map one-to-one to builds either: if a build sits in the queue, Jenkins rolls several build requests into one build, so one build can list several causes.

The deployment record. On Kubernetes, a Deployment gets a new revision only when its Pod template changes, for example when the container image is updated. Scaling does not create a revision. Each ReplicaSet name carries a hash of its Pod template, and the same hash goes on the Pods it creates as the pod-template-hash label. Any kubernetes.io/change-cause annotation is copied onto the revision when it is created. Deployment status conditions also carry lastTransitionTime stamps. During an update, the controller scales the old ReplicaSet down and the new one up, so a rolling update has a stretch when both versions are answering requests. A build finishing is not the moment the build starts serving. The gap between the two is propagation delay, and it belongs in the margin.

The inference log. The KServe inference logger emits CloudEvents for both the request and the response, each with a type, source, unique id and time. KServe presents it as a tool for debugging, auditing and monitoring models in production. The key question for the join is whether each logged response names a model version or Pod, or only the endpoint. With a version field, the join is on identity. With only a timestamp, it is on time alone, and every request inside a rollout window stays ambiguous.

Can Jenkins build timestamps and inference logs be correlated reliably enough for an audit?

Only once three conditions hold: the clocks were synchronised, the logs could not have been altered, and the artefact’s identity is confirmed by a hash rather than inferred from timing. The arXiv audit-trail paper makes the underlying point: audit trails show that events happened in a verifiable sequence, not that one caused another. Applied here, the records can place a build’s deployment in time, but without a hash match they cannot show that the binary answering requests is that build.

Clock synchronisation comes first, because inaccurate host clocks make event order untrustworthy. NIST SP 800-92 warns that inaccurate host clocks can make logs show event A 45 seconds before event B when A actually happened two minutes after B. NIST recommends NTP servers to keep log sources’ clocks consistent with each other.

Log integrity comes second. Jenkins logs have low integrity: an attacker with root access can edit logs and change timestamps, and the mitigation is to ship logs straight to a remote, append-only SIEM. Jenkins’s Audit Trail plugin, the usual source for “who changed what”, is not installed by default. Recording why a build started also needs a separate “Log build triggers” option turned on. If neither was set up during the window in question, the Jenkins side of the join rests on records an auditor can fairly question.

Artefact identity, confirmed by hash, comes third. Jenkins fingerprints a file by taking its MD5 checksum and records which builds of which projects used it. From a build page, “see fingerprints” lists every fingerprinted file and where it came from. NIST notes that log sources often record only what they consider most important, so they may share no common values, which makes linking events across sources hard.

How wide is the margin of error in a timestamp backfill, and how do you document it?

The margin at each boundary of a serving window comes from several separate error sources. RFC 5905, the NTPv4 specification, defines offset, round-trip delay, dispersion and jitter for each clock measurement. It defines the root synchronisation distance, LAMBDA = EPSILON + DELTA / 2 (root dispersion plus half the root delay), as the maximum error due to all causes. Dispersion grows at the clock frequency tolerance PHI, typically 15 ppm. Worked through, 15 parts per million is 15 microseconds per second, or about 54 milliseconds per hour (15 µs × 3,600). So on a host that stopped getting updates, dispersion, and with it the error bound, keeps growing each second until synchronisation resumes. RFC 5905 also uses a zero value to mean unknown or unsynchronised time. AWS reports that EC2 instances on the enhanced Amazon Time Sync Service typically show a clock error bound under 100 µs over NTP and under 40 µs over the PTP hardware clock. It also gives the bound as the chrony system time offset plus half the root delay, plus root dispersion, plus the PHC error bound.

In practice, that changes where to focus. On EC2 hosts using the enhanced Time Sync Service, microsecond clock bounds are tiny next to Jenkins queue waits, which the Jenkins documentation illustrates with a one-hour example. On a host with no sync record at all, skew is unbounded, and no amount of care elsewhere fixes that.

Error components to bound and record

Component	Where it comes from	How to bound it	What the auditor sees
Clock error, per host	NTP offset and dispersion on CI, cluster and serving hosts	Root synchronisation distance from chrony or ntpd records at the time	The bound for each host, or “unsynchronised: order not guaranteed”
Build timing	Jenkins scheduled time vs executor start	Join on executor start plus duration, never the scheduled time	Which Jenkins field was used
Propagation delay	Time from build completion to rollout start	Deployment events and lastTransitionTime stamps	The interval between build ready and rollout start
Rollout overlap	Old and new ReplicaSets serving at the same time	Rollout start to rollout complete	Requests in this interval marked ambiguous unless logged with a version
Logging granularity	Timestamp resolution in each log	The coarsest resolution among the sources joined	The resolution assumed

Writing it down for an auditor means the method goes in with the result. List the sources used and their retention status. Give each window boundary as an interval, not a point. Say which links rest on hash matches and which rest on time alone. Give each segment a confidence label.

There is also a point to stop. Suppose the audit needs one specific request attributed to a version, the request falls inside a rollout overlap, and the inference log has no version field. Timestamp correlation cannot settle that. The honest answer is to record the request as ambiguous and fix the logging from then on.

Can lineage be backfilled for models already running in production?

Yes, but only partially. Lineage for models already in production can be partially backfilled from deployment timestamps, CI/CD build records and inference logs. The record will have gaps and timing and correlation errors, which must be disclosed. A backfill is only as strong as the weakest record that survives.

TechnoLynx starts with an inventory of legacy models lacking lineage and their surviving records, then delivers a lineage record per model with unrecoverable fields marked, each entry cross-checked against artefact hashes and deployment manifests, plus gap-disclosure procedures for audit responses.

Before any of that, find out what has already gone. Several defaults and retention limits remove records a backfill needs:

• Kubernetes revision history. A Deployment keeps 10 old ReplicaSets by default, and the rest are garbage-collected. Each revision’s configuration lives in its ReplicaSet, so once that ReplicaSet is deleted, the record of that revision is gone.
• GitHub Actions audit streams. Standard audit log retention is 90 days, and git events are often only retrievable for 7 days.
• Regulated log retention. Under Article 19 of the EU AI Act, providers of high-risk AI systems must keep automatically generated logs for at least six months, unless other Union or national law says otherwise. Six months is a floor, so Article 19 alone does not guarantee that logs older than that still exist.

Preserve what is left before you parse anything. NIST SP 800-92 suggests that when logs may be needed as evidence, organisations keep copies of the original log files as well as the centralised and interpreted versions, in case the fidelity of copying or interpretation is questioned. It also describes storing a message digest of each archived log file so that any later change can be detected.

What is the minimum evidence for a regulated customer’s audit when lineage can’t be fully rebuilt?

What follows is our judgement, not a regulator’s threshold. The floor we would work to is a documented, timestamped artefact trail that ties the running model binary to a specific build, ties that build to a training run and an approval step wherever those survive, and states every gap rather than inferring or filling it. For comparison, one practitioner write-up on regulated lineage lists approval chain, deployment history and runtime linkage (tying a prediction to the model version that produced it) among the elements a practical lineage record should capture.

Checklist for a partial-lineage audit response

• The running artefact is identified by hash (image digest or file fingerprint), not by tag or name.
• That hash is matched to a CI build number and its executor start time.
• The build is linked to a training run and dataset version, or the link is marked unrecoverable.
• An approval or sign-off record exists for the release, or its absence is stated.
• Every serving window is given as an interval with its error components listed.
• The reconstruction method, its sources and their retention status are documented.
• Original log copies and their digests are preserved.

Some gaps should not be documented and left open. If the auditor needs something that cannot be recovered, such as the training data version, and the regulator will not accept that gap staying open, the fallback is fresh validation or retraining evidence, not a better story about the logs. The pre-instrumentation period should be labelled best-effort throughout. As a HackerNoon analysis puts it, retrofitting audit replay by rebuilding historical state from logs, backups and pipeline code is possible in principle, but its fidelity cannot be verified in practice, so its evidentiary standing is contested every time it is invoked.

What should a release pipeline capture so lineage never has to be backfilled again?

Capture lineage at release time, as a condition of shipping, rather than collecting logs and hoping they line up later. A practitioner write-up on regulated lineage names the most common failure as treating lineage as a logging afterthought instead of a structural constraint. The gaps then appear where pressure was highest, such as a release shipped on a deadline. Collecting a lot of logs does not fix this, because the same source notes that teams who over-collect often find in an audit that none of it can be queried in the shape the regulator asks for.

The arXiv audit-trail framework gives a concrete event model. In the framework, CI/CD steps, model registries and serving configuration managers are instrumented to emit DeploymentStarted, DeploymentCompleted, RolloutChanged and ServingConfigChanged events that capture runtime configuration and rollback actions. Its record format provides for an RFC3339 timestamp, scoped identifiers and integrity metadata that makes changing or deleting a record detectable. In the paper’s proof of concept, each entry stores a SHA-256 digest over its payload plus the previous entry’s hash. The HackerNoon analysis goes a step further and signs a hash of a state-complete record at decision time. It argues that the signing key should sit with a hardware security module or key management service that the governance function has independently validated, not in the same infrastructure that hosts the events.

On top of that event model, four additions are specific to model serving:

1. Write a model version or artefact digest into every inference response log, so the decision-to-version link is an identity join rather than a time join.
2. Record each host’s clock sync state alongside its logs. AWS’s open-source ClockBound daemon, for example, exposes a confidence interval for clock accuracy.
3. Stream CI/CD audit logs to an external SIEM, which the earlier source on GitHub Actions and Jenkins forensics says enterprise forensics needs given GitHub Actions’ retention limits and Jenkins log mutability. Then set Kubernetes revision retention, or export revisions, to match your audit horizon.
4. Write a retention policy that rules out the gaps you just found, so the next audit does not hit the same holes.

We treat all of this as release-readiness discipline. It belongs to the same engineering that produces regression suites and release gates, and it is separate from model accuracy. Building it into the delivery pipeline itself is MLOps engineering work, and the test of whether it is finished is practical. The regulated-lineage write-up suggests running a mock audit and timing how long it takes to trace one production decision back to its training and approval history.

Who can rebuild lineage for our deployed models before the audit?

TechnoLynx rebuilds lineage for models already in production and documents its gaps for audit responses, under Production AI Reliability.

This work has clear limits. It produces the engineering reconstruction and its gap documentation. If your records are intact and the audit question is narrow (one window, one model, version fields already in the logs), your own platform team can often answer it directly. What a particular regulator will accept is a question for your compliance function or counsel, not your engineers.

FAQ

What does “build” mean in Jenkins?

In Jenkins, a build is one run of a job. It has a unique build number, a scheduled timestamp, an executor start time, a duration in milliseconds and the causes that triggered it.

Is Jenkins a CI or a CD tool?

Jenkins is a general automation server, and teams use it for both continuous integration (building and testing on each change) and continuous delivery or deployment (pushing artefacts to environments).