GxP in pharma is the collective name for the good-practice rules covering manufacturing, clinical, laboratory, distribution and pharmacovigilance work (GMP, GCP, GLP, GDP and GPvP). Whether an AI tool counts as GxP depends on what its output is used for, not on the technology inside it. A tool whose output directly affects patient safety, product quality or data integrity falls within the regulators’ draft AI frameworks. FDA’s January 2025 draft guidance excludes tools that only draft submission text or support drug discovery, where that use does not affect patient safety, drug quality or the reliability of study results. The EU AI Act applies its own risk-based test on top of this. Its high-risk obligations now start on 2 December 2027 for Annex III systems and 2 August 2028 for Annex I systems.
Those are two different questions with two different owners. The rest of this page takes them in order, starting with GxP classification, then the AI Act, then the engineering work both of them depend on.
GxP in pharma, briefly
GxP refers to the good-practice laws, rules and guidelines for organisations that develop, test, manufacture, market or distribute products used by humans, with the “x” standing for the area in question. For AI the important part is not the list of acronyms but what every GxP regime shares. EUPATI describes that common ground as traceability (reconstructing the full history of a product and every work item behind it), accountability (knowing who contributed, when and how) and data integrity under ALCOA+ principles. Our explainer on GxP regulations and what they mean for AI and software covers each regime in more depth.
Once a company uses a computerised system for a GxP process, that system has to be validated and operated appropriately for its intended use. GAMP 5 Second Edition treats AI/ML as a subsystem within a larger computerised system.
Is our AI tool GxP or non-GxP?
The deciding question is what the output touches. The MHRA’s data integrity guidance puts it this way: data criticality depends on how the data influences quality, safety and efficacy decisions. FDA’s January 2025 draft guidance on AI for regulatory decision-making draws the same line. It applies to models that produce data supporting decisions on safety, effectiveness or quality, and explicitly excludes drug discovery, operational efficiencies and the mechanics of drafting submissions where those uses do not affect patient safety, drug quality or the reliability of study results. That guidance remains a draft with finalisation pending.
On the EU side, draft Annex 22 applies where AI models are used in critical manufacturing applications with direct impact on patient safety, product quality or data integrity, for example to predict or classify data. EMA’s reflection paper advises developers to seek early regulatory support when an AI system may affect a medicine’s benefit-risk balance.
MHRA notes that simple instruments with no configurable software may need only calibration, while complex systems need validation for intended purpose, with effort rising with complexity and risk. The draft revision of Annex 11 asks for Quality Risk Management across the whole lifecycle, weighted by process complexity, the level and novelty of automation, and the impact on product quality, patient safety and data integrity.
Worked classification examples
The table below applies those tests to typical uses. The rows are our reading of the frameworks cited above, not regulator rulings, and your QA function makes the final call.
| AI use | Output touches | Likely classification | Validation depth |
|---|---|---|---|
| LLM drafting sections of a submission, with authors editing the text | Document wording only | Outside FDA’s draft scope where it does not affect safety, quality or study reliability | Risk-based, set by your QA function |
| Target identification model in early discovery | Research hypotheses | Excluded from FDA’s draft framework; possibly inside the AI Act’s R&D exemption | Risk-based, set by your QA function |
| Classifier that passes or rejects units in visual inspection | Batch quality decisions | Critical GMP, inside draft Annex 22 scope | Full validation: intended use, acceptance criteria, independent test data |
| LLM summarising deviation records for a reviewer | GMP records, with a human deciding | Non-critical GMP use; draft Annex 22 says qualified personnel should be responsible for outputs | Risk-based, with human-in-the-loop review of outputs |
| Model whose predictions feed clinical or quality data in a filing | Data supporting regulatory decisions | Inside FDA’s draft credibility framework | Risk-based credibility evidence |
For the wider software question (spreadsheets, LIMS, MES and how GxP classification works across a system landscape), see our page on GxP vs non-GxP systems classification.
What draft Annex 22 allows in critical GMP use
It covers machine learning models that obtained their behaviour through training on data, and only static models that do not adapt during use. Dynamic models that keep learning in operation are outside its scope and should not be used in critical GMP applications. Generative AI and large language models are also outside it and should not be used in critical GMP applications either; where they are used in non-critical GMP work, qualified personnel should always be responsible for checking that outputs suit the intended use.
The text remains a draft consultation position. It was published for stakeholder consultation in 2025 alongside revisions to Chapter 4 and Annex 11, drafted by the EMA GMDP Inspectors Working Group with PIC/S. Elementum reads the consultation draft as also proposing mandatory explainability for AI models.
What the draft expects of a model in scope is concrete:
- Intended use and input space. The intended use has to be described in detail, including the full range of expected inputs with their common and rare variations, known limitations and possible biased inputs, and a process subject matter expert approves that description before acceptance testing.
- Acceptance criteria. The model must perform at least as well as the process it replaces, which means the current process performance has to be known.
- Test data independence. Technical or procedural controls must keep test data out of development, training and validation, for example by splitting it off before training starts.
- Ongoing oversight. Annex 22 foresees change control, model performance monitoring and human review procedures throughout operation.
- Shared ownership. SMEs, QA, data scientists, IT and consultants work together across selection, training, validation, testing and operation, each with defined responsibilities and access.
The draft still allows generative models in non-critical GMP work under human oversight. GAMP 5 Second Edition treats AI/ML as a subsystem of a larger computerised system and supports placing probabilistic AI inside deterministic workflows: a workflow layer sequences the process, limits what each component can see and produce, and routes outputs through human review before anything becomes a controlled record. In our view, that wrapper is the most useful architectural decision a team can make early, because it turns “can we validate an LLM?” into “can we validate the controls around it?”.
Does the EU AI Act make pharma AI high-risk?
The AI Act defines four risk levels (unacceptable, high, transparency and minimal) and states that most AI systems in use in the EU fall into the minimal category, which it does not regulate. A system becomes high-risk through one of two routes in Article 6. The first applies when it is a product, or a safety component of a product, covered by the Union harmonisation legislation in Annex I and required to undergo third-party conformity assessment. The second applies when it falls under a use listed in Annex III. The Commission’s published Annex III examples cover areas such as critical infrastructure, education, employment, essential services, biometrics, law enforcement, migration and justice.
Article 6 also narrows the safety-component route. Systems used solely for non-safety aspects such as performance optimisation, automation or quality control do not qualify as safety components, unless their failure would endanger health and safety. An Annex III system is not high-risk when it poses no significant risk of harm and meets a listed condition, such as performing a narrow procedural or preparatory task, although profiling of natural persons always stays high-risk. A provider relying on that derogation must document the assessment before placing the system on the market and remains subject to registration.
So classification is case by case, and the answer depends on the specific tool.
The R&D exemption is an argument, not settled law
Article 2(6) excludes AI systems developed and put into service for the sole purpose of scientific research and development, and Article 2(8) excludes research, testing and development before a system is placed on the market, though testing in real-world conditions is not covered. EFPIA argues that the exemption covers AI-based drug development tools, and that even without it most AI in medicines R&D cannot legally be high-risk because it falls under neither Annex I nor Annex III. Pinsent Masons notes that EFPIA’s position rests on its own interpretation of the high-risk scope. That is a reasonable reading, but it is a reading: we would plan for it while keeping the evidence that a counter-reading would demand.
Dates after the AI Omnibus
The AI Act entered into force on 1 August 2024 and became generally applicable on 2 August 2026, with exceptions. The AI Omnibus moved the high-risk chapter: classification rules, requirements and operator obligations apply from 2 December 2027 for Annex III systems and from 2 August 2028 for Annex I systems. The original 2 August 2026 date for high-risk rules is no longer the operative one. Article 6 set 2 February 2026 for the Commission’s classification guidelines; under the Omnibus the deadline is 1 August 2027.
If a system is high-risk, much of the work overlaps with GxP
Article 9 requires a risk management system run as a continuous process across the lifecycle, with testing against predefined metrics and probabilistic thresholds suited to the intended purpose. It explicitly allows those steps to be combined with risk management procedures required under other Union law. Article 12 requires logging that supports traceability appropriate to the intended purpose, including events relevant to risk, substantial modification and post-market monitoring. The Commission lists high-risk obligations as risk assessment and mitigation, dataset quality, activity logging, detailed documentation, deployer information, human oversight, and robustness and cybersecurity. We would build one evidence base and map it to Annex 11, Annex 22 and the AI Act rather than run two programmes.
The frameworks that already govern AI today
Teams waiting for final AI rules are waiting for the wrong thing, because existing pharmaceutical regulatory compliance frameworks already apply. The draft revised Annex 11 would apply to all computerised systems used in manufacturing medicinal products and active substances, would require validation before use and a maintained validated state, and would require a user requirements specification whether the system is built in-house, bought off the shelf or used as a service. FDA’s 21 CFR Part 11 requires secure, computer-generated, time-stamped audit trails for electronic records. GAMP 5 Second Edition, including its Appendix D11 on AI/ML, is the primary route to validating AI tools for GxP use, scaled to risk and covering intended use, model behaviour, change control, performance monitoring, human oversight and traceability. Across all of this, the joint EMA and FDA principles for good AI practice span the medicines lifecycle from early research to manufacturing and safety monitoring.
One clause in draft Annex 11 addresses vendor-qualified tools. Relying on a vendor’s or service provider’s qualification does not move responsibility: the regulated user stays fully responsible based on the risk to product quality, patient safety and data integrity.
ALCOA+ for training data, prompts and outputs
ALCOA+ stands for attributable, legible, contemporaneous, original and accurate, plus complete, consistent, enduring and available. When AI output becomes a GxP record, those requirements apply to it. For AI systems, ALCOA+ reaches further back than the final record, because the record cannot be reconstructed without the inputs that produced it.
MHRA expects user-defined processing parameters to be traceable to raw data, with audit trails that allow every processing step to be reconstructed, and with repeated processing under modified parameters kept visible so that nobody can tune their way to a preferred result. That last point matters for AI, since rerunning a prompt or a threshold until the answer looks right is exactly the pattern it describes. Elementum’s guidance for Part 11 contexts recommends a provenance record for each AI run that identifies the model and version, preserves the input prompt and retrieval context, and keeps the raw output with the reviewer’s action and the final committed record.
Data governance must apply across the whole data lifecycle, from generation through processing, use, retention, archive and destruction. Data quality means assurance that the data is what was intended and fit for purpose. The Commission’s consultation summary says draft Annex 22 puts emphasis on the quality of training data and the management of test data.
Change control and revalidation when the model is retrained
Draft Annex 22 covers static models only, so a retrained model is a changed system, not the same system continuing to learn. Change control means evaluating changes to decide whether they affect the validated state. Periodic reviews check whether the system is still fit for intended use and whether full or partial revalidation is needed, covering changes to software, configuration, platform, infrastructure and interfaces, and the combined effect of several changes. In practice the triggers we would wire into the pipeline are new model weights, a changed training or test set, a different base model or inference runtime, and drift in monitored performance, each opening a change record before deployment.
Engineering work and quality decisions are different jobs
A useful way to scope an AI-in-GxP project is to separate what engineering builds from what quality, regulatory and legal staff decide. Engineering produces the evidence. It does not make the decision.
| Activity | Who owns it | What the frameworks expect |
|---|---|---|
| Provenance record per AI run | Engineering | Model and version, prompt, retrieval context, raw output, reviewer action, committed record |
| Audit trail on data changes | Engineering | Who, what (old and new value), when with time zone, reason prompted at the time of change |
| Audit trail protection | Engineering, verified by QA | Audit trails on and not user-switchable; admin changes recorded |
| Access control | Engineering, approved by QA | Unique accounts, segregation of duties, least privilege |
| Test data independence | Engineering | Technical or procedural separation from training and validation |
| Performance monitoring and versioning | Engineering | Continuous oversight with change control and monitoring |
| Intended use and input space | Process SME | Documented and approved before acceptance testing |
| Acceptance criteria | QA and SME | At least as high as the process replaced |
| Go-live and conditional approval | Regulated user | Documented assessment that open deficiencies do not affect quality, safety or data integrity |
| AI Act classification or derogation | Provider, with counsel | Assessment documented before market placement |
Evidence that a non-engineer can read and sign against is itself an engineering output, not a policy document. Our page on what GxP compliance requires for AI software in pharmaceutical manufacturing goes further into the validation documentation side.
Who can build the logging and evidence layer for a GxP AI tool?
TechnoLynx builds the evidence that makes a GxP AI tool inspectable: task-specific evaluations, governance evidence packs aligned to named rubrics such as GxP references, and lineage and traceability for production decisions that QA can review and sign against. On the in-house question, a team that already runs validated systems and has ML engineers who understand inference pipelines can build this itself. A specialist earns its place when the model, the serving stack and the evidence trail have to be built together under a deadline. When choosing a partner, ask to see how it records a single AI run end to end and how it handles a retrain under change control. Our work for this sector is described on the AI engineering for life sciences and healthcare page.
The limits are firm. TechnoLynx builds this evidence as engineering, not as policy advice or certification; it produces the artefacts an audit needs. Classification under the AI Act belongs with counsel, and validation strategy, acceptance criteria and release decisions belong with your QA function. Companies that need computerised system validation documentation rather than engineering may be better served by a CSV specialist, and questions about the R&D exemption need regulatory lawyers.
The open question for most pharma teams
A batch-release classifier is plainly critical GMP, and a drafting assistant is plainly not. The difficult cases are tools that start as non-critical aids and gradually become something people rely on, such as a deviation summariser whose output reviewers stop checking. Draft Annex 22 makes qualified personnel responsible for outputs in non-critical use of generative models, and a per-run provenance record keeps the reviewer’s action with each output. If you instrument that from the first release, a reclassification later becomes a documentation exercise instead of a rebuild.