AI HIPAA compliance belongs to the workflow that handles protected health information (PHI). It does not come with the model, a HIPAA compliant vendor or the contract. A business associate agreement (BAA) is a contract, and signing one does not show that a product is configured or operated securely. An AI workflow becomes HIPAA-ready through engineering: you know where PHI enters and where it stays, you implement the Security Rule safeguards at each of those points, and you produce evidence that a privacy reviewer can follow one record at a time.
Our wider piece on why readiness is a property of the workflow, not a vendor label covers scope and labels. This page covers the engineering underneath.
What does HIPAA-ready mean for an AI workflow, as opposed to a signed BAA?
HIPAA-ready means every place PHI touches is covered by a safeguard you can show working. The BAA is only the legal basis that lets a vendor sit in that path. A BAA is needed whenever a vendor will create, receive, maintain or transmit PHI on your behalf, whether or not the product calls itself AI. HHS guidance on cloud computing says that a cloud provider which lacks the encryption key is still a business associate, still needs a BAA, and is directly liable under the HIPAA Rules.
A signed BAA settles less than the procurement checkbox suggests. A generic BAA can meet HIPAA’s required contract elements and still say nothing about model training, prompt retention, model subprocessors or change notifications, unless the parties add those terms. Compliance also depends on the vendor’s safeguards and on the healthcare organisation’s own configuration, access controls, policies, workforce training and ongoing oversight. NIST SP 800-66r2 states that each regulated entity is responsible for its own Security Rule compliance and violations.
There is also no government certification for HIPAA compliance. A vendor that calls itself “HIPAA certified” has been through its own assessment or a third party’s, not an HHS approval. So the real question is never “is this tool compliant?” It is whether this deployment of the tool, inside your data flow, meets the safeguards and can show that it does.
AI HIPAA compliance starts with a map of where PHI persists
NIST SP 800-66r2 (February 2024) says a regulated entity should know, before any risk assessment, where ePHI is created, received, maintained, processed and transmitted. That includes whether it goes to external parties such as cloud service providers. In an AI workflow this map is wider than the clinical database. Every prompt, cache and log line that holds a fragment of a patient record belongs on it.
| Where PHI ends up | How it gets there | What has to be controlled |
|---|---|---|
| Prompts and retrieved context | The user pastes it, or a retrieval step pulls record chunks into the prompt | The model sees only what the calling user may see, not what the service account can reach |
| Provider abuse-monitoring logs | OpenAI generates these by default for all API use and keeps them for up to 30 days | A BAA-covered configuration, or approved Modified Abuse Monitoring or Zero Data Retention |
| Stored sessions | In OpenAI GPT-Live sessions with store: true, the completed session recording is kept for 30 days | Whether storage is enabled at all |
| Prompt caches | OpenAI’s extended prompt caching in regions without regional processing may store customer content outside the region for a time | Caching settings matched to the residency commitment |
| Your own gateway, telemetry and tracing | Request logging in your serving layer; if the gateway logs prompts in plaintext, the leak is in your own code | Redaction, encryption and retention limits on your logs |
| Third-party tools | Data sent to an MCP server falls under that server’s policies, and OpenAI’s live web search is not HIPAA eligible | An allow-list of tools in the covered path |
| Training, fine-tuning and evaluation sets | Prompts and outputs reused to improve a model | A contract clause banning training, fine-tuning, evaluation or model improvement on your PHI |
| Transcripts and backups | Retained copies of all of the above | Set retention periods and a process that certifies deletion |
Keeping these points under control without losing the record of what happened is a design problem of its own. Our guide to handling PHI without breaking the audit trail works through it.
Which Security Rule safeguards need engineering, not policy language
The Security Rule is built from standards. Many of them carry implementation specifications, and each specification is either required or addressable. Required means you must implement it. For an addressable one, you assess whether it is reasonable and appropriate in your environment. You then either implement it, or record why not and put in place an equivalent alternative. Addressable has never meant optional.
NIST explains the rule’s flexibility: the required standards and specifications are the same for every regulated entity, but a reasonable implementation can differ between organisations. A small medical practice and a national health plan both have to keep and examine audit logs, and their implementations will often look very different. HHS likewise allows any security measure that reasonably and appropriately implements the standards, and leaves the choice of technology to the covered entity. But each entity has to document the activities it decides are necessary and appropriate.
The right-hand column below is how we would put each safeguard into an AI serving path.
| Safeguard and citation | Required or addressable | How we would engineer it in an AI serving path |
|---|---|---|
| Unique user identification, under access control 164.312(a)(1) | Required | Each inference request traces back to a named person, not a shared API key |
| Emergency access procedure, 164.312(a)(1) | Required | A documented route to needed ePHI if the AI system or its upstream is down |
| Automatic logoff, 164.312(a)(1) | Addressable | Session timeouts on clinician-facing assistants |
| Encryption and decryption, 164.312(a)(1) | Addressable | Encryption at rest for vector stores, caches and log stores |
| Audit controls, 164.312(b) | Standard | Mechanisms that record and examine activity in every system that contains or uses ePHI, model endpoints included |
| Mechanism to authenticate ePHI, under Integrity 164.312(c)(1) | Addressable | A way to show that stored records and retrieved context were not altered without authorisation |
| Person or entity authentication | Standard | Verify identity before a user or a calling service reaches ePHI |
| Encryption, under Transmission security | Addressable | TLS on every hop: client to gateway, gateway to model, model to tools |
| Information system activity review, 164.308(a)(1) | Required | Someone regularly reviews the audit logs, access reports and incident tracking |
| Risk analysis and risk management, 164.308(a)(1) | Required | A documented risk assessment that covers the AI components |
Two of these deserve more attention than they usually get. The first is minimum necessary access. HHS administrative safeguards say each workforce member or job function gets only the ePHI it needs. The Privacy Rule’s minimum necessary standard asks policies to name who needs access, which categories of PHI they need, and under what conditions. In a retrieval-augmented system, this is enforced in the retrieval query, and it fails if the retriever runs with the service account’s broad permissions. The second is authentication. The current Security Rule does not name multi-factor authentication. It requires identity verification and treatment of the risks found in the risk analysis.
Can a third-party or hosted model API sit inside a HIPAA-ready workflow, and what has to be engineered around it?
Yes. A hosted model API can sit inside a HIPAA-ready workflow when the provider signs a BAA, you use only the configuration that BAA covers, and your own administrative and technical safeguards surround it. OpenAI’s documentation, for example, says that customers who have signed its Business Associate and Healthcare Addendum, and whose org is provisioned with Private Retention with Private Safety Processing or Safety Retention, can use BAA-eligible endpoints for PHI even when data is retained.
Zero retention does not replace a BAA. DeAI’s 2026 review of zero-retention APIs makes the point both ways: a provider promising zero retention is still unusable for PHI if it will not sign a BAA, while a signed BAA alongside a 30-day abuse log can work fine. Retention promises on a web page are policy statements, not facts you can verify, which is why the abuse-monitoring window and any zero-retention addendum need to be written into the contract.
Everything around the endpoint is where the engineering work sits:
- The covered surface. A 2026 analysis by Definite notes that a BAA covers a configuration narrower than the whole product. Enabling an excluded feature quietly takes the deployment out of coverage, so the allowed surface should be pinned in code review. OpenAI’s live web search is one such exclusion.
- Every hop. Each party that handles PHI on your behalf needs a BAA, including the cloud provider and any vendor between you and the model.
- Your side of the logs. Access controls, audit logging and minimum necessary data in each prompt stay your responsibility.
- Residency claims. On OpenAI’s platform, regional storage does not imply regional processing.
- The contract additions. These are a training ban, a subprocessor list with change notice, retention and deletion terms covering prompts, transcripts, logs and backups, and assessment rights.
Self-hosting open-weights models in your own VPC takes the model provider out of the chain. It also leaves you owning the whole serving stack’s Security Rule story: TLS termination, authentication, request-level audit logging and patching. For verdicts on specific assistants, our page on whether ChatGPT is HIPAA compliant goes through them vendor by vendor.
How does de-identification change what the workflow has to control, and where does re-identification creep back in?
De-identification shrinks what the workflow has to control. Information that meets HIPAA’s de-identification standard is no longer protected health information, so the controls above stop applying to that dataset. HHS recognises two ways to meet the standard: Expert Determination and Safe Harbor. HHS guidance also covers free-text fields and what counts as “any other unique identifying number, characteristic, or code”.
The trouble is that a model’s outputs can carry the PHI in its inputs into derived data. A study that generated synthetic clinical notes from MIMIC-III records with GPT models and Mistral 7B restored PHI to its de-identified source notes from a mapping file, then found every PHI category except Social Security numbers in the synthetic notes. Generation method made a difference to PHI exposure. One-shot generation carried the highest exposure, and keyword-based generation the lowest, mostly limited to geographic terms. With one-shot generation on Mistral 7B, geographic PHI appeared at 85.35%, dates at 87.45% and unique identifying numbers at 10.40%. GPT-4 and Mistral 7B each produced about 500 notes in which two types of PHI appeared together. The authors warn that such co-occurrence can link a note directly back to the original patient. Unique identifiers such as medical record numbers and device identifiers could be extracted whole and reappear in outputs.
The study ran its GPT models on an Azure OpenAI platform it describes as HIPAA-compliant, and the outputs still carried PHI.
In engineering terms, we would treat anything a model produces from PHI-bearing input as PHI until a scan of the output shows otherwise. That includes synthetic datasets, summaries, evaluation sets and fine-tuning corpora. The generation method then becomes a control in its own right, and output scanning becomes evidence. If a vendor wants to use de-identified data to improve its product, the contract should state the de-identification method, who applies it, and whether you allow the use at all.
What audit evidence does a privacy reviewer expect the workflow itself to produce?
A privacy reviewer expects the workflow to produce records showing that activity on ePHI is captured and that someone examines it. The audit controls standard at 45 CFR 164.312(b) requires mechanisms that “record and examine activity” in systems that contain or use ePHI. The examining part is a requirement of its own: information system activity review requires procedures for regularly reviewing audit logs, access reports and security incident tracking reports. The risk analysis decides how detailed those controls are. For an LLM workflow, the log has to answer who prompted what, when, and what came back. That makes your inference logs and their retention settings compliance artefacts.
This kind of evidence is an engineering output in its own right. It is the artefact a compliance owner or auditor reads and signs against, and it is produced as engineering, not written as policy text.
Evidence checklist for a HIPAA review of an AI workflow
- A current map of where ePHI is created, stored and transmitted, with third parties named.
- A documented risk analysis, plus a risk register listing threat and vulnerability pairs, remediation plans, timelines and owners.
- A recorded decision for each addressable specification: implemented, or an alternative with the reasons.
- Request-level audit logs tied to unique user identities, retained under your policy and queryable on demand.
- Procedures for reviewing the logs regularly, not just collecting them.
- Signed BAAs for every party in the PHI path, with the AI-specific terms attached.
- Records of security incidents and their outcomes.
- Periodic evaluations repeated after environmental or operational changes. In our view, swapping a model or enabling a new endpoint feature counts as such a change.
- Output leakage test results wherever de-identified or synthetic data feeds a model.
Under both the current rule and the proposed one, “we do it but didn’t write it down” is a finding. For the build side, see our pieces on audit-ready model documentation and lineage, on designing an AI workflow with the audit evidence built in, and on what an evidence pack contains, section by section.
What falls outside engineering audit-readiness?
Legal judgement falls outside engineering audit-readiness. No one can hand over a HIPAA certificate either, because none exists. The line between workflow readiness and certification is worth agreeing internally before any engagement begins.
Scope is the other thing to settle before engineering starts. Regulation is often less constraining than teams assume. Plenty of AI work in a life sciences or healthcare company never touches PHI, and checking the real scope first avoids building controls nobody needed. Where PHI flows, though, the Security Rule applies to the whole path.
On the engineering side, TechnoLynx builds the evidence a reviewer works from. That means governance evidence packs aligned to HIPAA references, plus lineage and traceability for production decisions. All of it is engineering work, not policy advice or certification. Our AI engineering work for life sciences and healthcare covers this evidence work.
The rule itself may change. In January 2025, HHS published a proposed update to the Security Rule in the Federal Register (90 FR 898). It would remove “addressable” safeguards and make MFA and encryption mandatory, along with a written asset inventory and network map. As of October 2026, according to BeefedUp IT, the proposal was still pending, with final action targeted for July 2027. Until a final rule is published, the current split between required and addressable stands. A workflow built today to the current rule, with its decisions written down, is easier to move to the new rule if it lands, because the flow map and written decisions already exist.
FAQ
What are the three types of HIPAA safeguards?
The Security Rule defines administrative, physical and technical safeguards. Technical safeguards are the technology, and the policies and procedures for its use, that protect ePHI and control access to it.
Is training a technical safeguard?
No. Training requirements sit among the administrative safeguards, alongside the security management process and documentation. Access control, audit controls and integrity are technical safeguards under 45 CFR 164.312.
