An auditor will not ask whether your vendor is HIPAA compliant. They will ask you to show what happened to one specific patient record as it moved through your pipeline — which system touched it, who could retrieve it, how long it persisted, and where the log entry proving that is. A vendor certificate cannot answer that question, because the question is about your workflow, not their service.
This is the most common failure pattern we see when procurement hands an ML team a signed BAA and a HIPAA attestation. The document is real, the vendor’s controls are real, and the team reasonably concludes the compliance problem has been solved upstream. It has not. A vendor attestation is a boundary statement: it describes what the vendor’s service does under the vendor’s own configuration assumptions, and it stops at the edge of that service. Everything your pipeline does on your side of that edge is yours to design, operate and evidence.
What does a vendor HIPAA attestation actually cover?
It covers the vendor’s infrastructure, their internal access controls, their encryption posture, their subcontractor chain, and their contractual obligation to notify you of a breach. That is genuinely useful and you should require it.
What it does not cover is the shape of the data you send. If your inference service embeds a patient identifier in a prompt, that identifier is PHI in your request payload before it reaches the vendor and in your application logs after the response returns. If your computer-vision pipeline decodes clinical video into frames for a model, those frames are PHI sitting on a worker node’s scratch disk. If you cache embeddings to cut latency, that cache is a PHI store. None of these live inside the vendor’s boundary. All of them will be in scope during a review.
The other quiet limitation: attestations are point-in-time. A vendor is compliant as assessed on a date under a configuration. Your workflow drifts — a new retention default, a debug log level left on after an incident, a service account granted broad read for a migration and never revoked. The certificate does not track that drift, and neither does anything else unless you build monitoring for it. That is why we treat compliance posture as an operational property with an owner, not a document in a folder.
The control map: vendor boundary versus workflow side
The single most useful artefact at the start of a regulated AI engagement is a two-column map of where each control actually lives. It takes a few hours to produce and it reframes the whole conversation, because it converts “we’re covered” into a list of named gaps with owners.
| Control | Vendor boundary | Your workflow side |
|---|---|---|
| Encryption in transit | TLS to the vendor endpoint | Internal service-to-service hops, message queues, object stores |
| Encryption at rest | Vendor-managed storage | Intermediate artefacts: decoded frames, temp files, embedding caches, model checkpoints containing training PHI |
| Access control | Vendor console roles, API keys | Who can read the S3 bucket, exec into the pod, query the feature store, pull a Docker image with baked-in test data |
| Audit logging | Vendor-side API access logs | A reviewable record of which human or service accessed which patient record, when, and why |
| Retention | Vendor’s stated retention for requests | How long your payloads, caches, logs, and reprocessing queues hold PHI — and whether deletion is enforced or aspirational |
| Lineage | Not offered | Which model version, which input record, which output influenced a clinical or GxP-relevant decision |
| De-identification | Almost never the vendor’s job | Whether PHI needed to enter the payload at all |
The row that fails most often in practice is audit logging. Teams have logs — application logs, request traces, Kubernetes events — but not a record structured around patient record accessed by principal at time for purpose. Observability tooling and audit logging look similar and serve different readers. One is for your on-call engineer; the other is for a reviewer reconstructing a specific access event two years later.
Where GxP makes the same mistake differently
In a GxP context the vocabulary changes but the structure of the failure is identical. A vendor supplies a qualified system; qualification is a supplier-level statement. Validation is about the system as you use it — your configuration, your intended use, your data flows, your acceptance criteria. Vendor qualification is an input to validation, never a substitute for it.
The awkward part for AI workflows is that “as you use it” is a moving target. Retraining, prompt changes, a swapped embedding model and a new inference runtime all change the system as used. If your validation package assumes a static component, the first model refresh silently invalidates it. Naming which changes trigger revalidation — and instrumenting the pipeline so those changes are detectable — is engineering work, not paperwork.
Does a compliant API endpoint change your obligations for the payloads?
No. Routing PHI or clinical video through a compliant endpoint changes who is responsible for the endpoint. It does not change your obligations for what you assembled to send, what you received back, what you cached along the way, or what your framework wrote to disk while doing it.
This matters most in vision and multimodal pipelines, where the intermediate state is large and easy to overlook. A typical deployment decodes video with OpenCV or a hardware decoder, writes frames to a scratch volume, batches them into a PyTorch or ONNX Runtime inference step, and pushes results downstream. Two of those stages produce PHI artefacts that nobody explicitly decided to create. In our experience they are found during design review roughly as often as they are found by the team that built the pipeline — which is to say, not reliably either way, which is exactly why the map exists.
What the remediation cost looks like
The reason we push this to the front of an engagement is sequencing, not diligence theatre. Retrofitting audit logging and retention enforcement into a pipeline already in clinical or GxP use means change control, revalidation, and often reprocessing historical data to establish a lineage record that was never captured. Designing the same controls before deployment is ordinary engineering. Across the regulated engagements we have supported, the retrofit path consistently costs a multiple of the designed-in path — that is an observed pattern from our own project work, not a benchmarked figure, and the multiplier depends heavily on how much of the pipeline is already frozen under change control.
The measurable outcome to aim for is simple: the number of control-coverage gaps identified on the workflow side before a review rather than after. Encryption at rest for intermediate artefacts. Audit-log completeness for PHI access. Retention actually enforced. Lineage for model inputs and outputs. Each gap closed pre-deployment is one that does not become a finding on a running system.
For the broader picture of how regulated AI programmes structure this work — including where a monitoring harness and a validation package fit alongside the control map — see our coverage of building AI systems for life sciences.
One boundary worth stating plainly about our own role: this is engineering audit-readiness, not legal compliance certification. We map controls, build the evidence trail, and instrument the pipeline so drift is visible. Whether that satisfies your regulator is a determination your compliance and legal functions make, with the trail we produce as their input.
The question to take back to your next architecture review is not whether your vendor is compliant. It is whether, given a single patient identifier, someone on your team could reconstruct that record’s full path through your system tomorrow morning — and prove it.
Frequently Asked Questions
What does it mean in practice that a “HIPAA compliant vendor” doesn’t make your AI workflow compliant?
Looking at Why a “HIPAA Compliant Vendor” Doesn’t, it means the vendor’s attestation describes their service under their configuration, and stops at the edge of that service. Your prompts, payloads, decoded frames, caches, logs, retention behaviour and access scoping sit outside that edge and are yours to design and evidence., it covers the vendor’s infrastructure, internal access controls, encryption posture, subcontractor chain and breach-notification duty. Coverage stops at the boundary of their service — and it is point-in-time, so it says nothing about drift in your workflow after the assessment date.
Which workflow-side controls are most often assumed to be inherited but are not? Encryption at rest for intermediate artefacts, audit logging structured around patient-record access, enforced retention, access scoping on buckets and pods, and lineage linking model inputs to outputs. Audit logging fails most often, because observability logs are mistaken for audit records.
What does an auditor ask for that a vendor certificate cannot answer? They ask you to trace one specific patient record through your pipeline: which systems touched it, which principals could retrieve it, how long it persisted, and where the log entry proving each step is. A certificate is a statement about a supplier; this is a statement about your system’s behaviour.
How does the same gap appear under GxP? Vendor qualification is a supplier-level statement; validation covers the system as you actually use it, with your configuration and intended use. For AI workflows the difficulty is that retraining, prompt changes and runtime swaps move the “as used” target, so revalidation triggers must be named and made detectable.
Does routing PHI or clinical video through a compliant API endpoint change your obligations for the payloads, caches and logs on your side? No. It changes who is responsible for the endpoint. What you assembled to send, what came back, what your framework wrote to a scratch volume, and what your cache retained all remain your responsibility.
How do you map the vendor boundary against your workflow before a review, and what evidence do you produce for the workflow side? Build a two-column control map — vendor boundary versus workflow side — across encryption, access, logging, retention, lineage and de-identification, then assign an owner to every workflow-side row. The evidence is the trail itself: access records, retention enforcement, and lineage from input record to model output.
Three Steps to Close the Compliance Gap
Your organization remains the covered entity regardless of vendor certifications, which means documenting how PHI flows through your specific AI implementation cannot be outsourced.
