An AI tool’s regulatory class is set by what it is intended to do, not by the model underneath it. Medical device regulators classify software by its intended purpose, wherever it runs, and US bank model-risk guidance under SR 11-7 decides scope by use case, so the same LLM can be in scope for one application and out of scope for another. For a transplant-support tool and an insurance chatbot alike, the intended use drives the classification and the evidence that follows from it.
TechnoLynx can help a team in that position check whether its intended-use statement and supporting evidence would stand up before the tool goes live, and the AI Readiness Scorecard is the first step: an assessment of where the documentation, testing and oversight evidence stands against what the chosen classification implies.
Intended use, not actual use, decides whether software is a medical device
The deciding question is what the software is for, not what someone happens to do with it. A general-purpose video conferencing application is not Software as a Medical Device (SaMD) even when clinicians use it for telemedicine, while a platform built for remote patient monitoring with clinical decision support is, because its intended purpose is medical. The International Medical Device Regulators Forum (IMDRF) defines intended use as the manufacturer’s objective intent, as reflected in its specifications, instructions and information. The EU Medical Device Regulation (MDR) casts the net wider: intended purpose comes from the label, the instructions for use, and promotional or sales materials and statements.
A sales deck that says the software “decides” undoes a careful label that says it “informs”.
How the intended-use statement sets the risk class
Under the IMDRF SaMD framework (N12, 2014), the intended use carries two factors that set the category: how significant the software’s information is to the healthcare decision, and the state of the healthcare situation or condition. Significance comes in three levels (treat or diagnose, drive clinical management, inform clinical management), and the condition is critical, serious or non-serious. The combination gives four categories, with Category IV the highest impact and Category I the lowest. Categorisation, N12 says, relies on an accurate and complete SaMD definition statement.
The EU applies the same logic through Rule 11 of MDR Annex VIII. Software that provides information used for diagnostic or therapeutic decisions is Class IIa, rising to Class IIb where those decisions could cause a serious deterioration of health or a surgical intervention, and to Class III where they could cause death or an irreversible deterioration; other software is Class I. The European Commission’s MDCG 2019-11 guidance points manufacturers to the IMDRF table as a useful indication of the class Rule 11 produces.
Significance and condition mapped to IMDRF category and EU class
| Significance of the information | Condition | IMDRF category | EU MDR class |
|---|---|---|---|
| Treat or diagnose | Critical | IV (acute stroke image analysis example) | III |
| Treat or diagnose | Non-serious | II | IIa |
| Drive clinical management | Critical | III | IIb |
| Drive clinical management | Serious | II | IIa |
| Drive clinical management | Non-serious | I | IIa |
| Inform clinical management | Critical | II | IIa |
| Inform clinical management | Non-serious | I | IIa |
The IMDRF categories in the table come from N12’s worked examples and MDCG 2019-11, and the EU classes from Hardian Health’s mapping. The table is an orientation, not a substitute for applying Rule 11 to the actual intended purpose.
Two things in the table deserve attention. IMDRF categories and EU classes are different scales, and conflating them with each other or with FDA classes is one of the common ways SaMD gets underclassified. The other is that class is not a quality grade: Class I devices in the US and EU are often self-certified, whereas for Class II and III a regulator or notified body audits the clinical and technical evidence.
When transplant-release software lands in the top category
Software that informs an irreversible clinical decision, such as releasing an organ for transplant, sits at the top of both frameworks unless the intended use genuinely limits its role. Rule 11 puts software in Class III where decisions based on incorrect information are reasonably likely to cause death or an irreversible deterioration of health. On the IMDRF side, software that drives clinical management in a critical situation is Category III, and software used to treat or diagnose in a critical, time-sensitive condition is Category IV.
The question is often put as “is this Class III under IMDRF?”, and that wording is worth correcting first: IMDRF assigns categories, not classes, and the class comes from the jurisdiction’s own rules. A team aiming for Category III rather than IV has to show that the software drives clinical management rather than treating or diagnosing.
Informing clinical management means the information will not trigger an immediate or near-term action. A release decision on a donor organ is immediate by nature, so an “inform” label would be contradicted by the clinical workflow itself. In the US, the FDA’s clinical decision support guidance (revised January 2026) does not treat software for a critical, time-sensitive task as meeting Criterion 4, because a clinician is unlikely to have time to review the basis of the recommendation independently. That means the CDS exclusion from device regulation is not available for such a tool. In one MDCG 2019-11 example, cognitive therapy software drops from Class III to IIa when a specialist, not the software, determines the therapy. That only works if the specialist really decides, which is the question the human-review section takes up below.
For the wider path from a validated model to a cleared device, our piece on medical device regulation and AI, where validation ends and clearance begins picks up where classification stops.
Intended use versus indications for use
In FDA usage, intended use is the general purpose or function of the device and encompasses the indications for use. Indications for use describe the disease or condition the device will diagnose, treat, prevent, cure or mitigate, including the patient population it is intended for. FDA determines intended use from the proposed labelling, with the indications for use statement as one factor.
A change in indications does not automatically create a new intended use; it does so only when it raises different questions of safety and effectiveness that block a meaningful comparison with the predicate device. FDA singles out the changes most likely to do that: functional to treatment indication, diagnostic to screening or back, a different anatomical structure, a different patient population such as adult to paediatric, and a different clinical setting such as periodic to continuous monitoring or hospital to home. A new intended use makes the device not substantially equivalent, and that result places it in Class III.
MDCG 2019-11 requires manufacturers to evaluate how changes to function or intended use affect qualification and class, and N12 requires a risk assessment of whether a change affects the category and the core functionality in the definition statement. A retrained model that now gets used on a new population is an indications change, whatever the release notes call it.
Writing an intended-use statement that holds up under review
A defensible statement is specific on the points the classification rules read. The statement should cover these elements:
- Significance of the output: whether it treats or diagnoses, drives or informs the decision.
- State of the condition: critical, serious or non-serious.
- Core functionality: the features essential to that significance in that situation.
- Intended user and population: who reviews the output and for which patients, as the FDA CDS guidance expects in labelling.
- Time-criticality: whether the user has time and training to review the basis of the output.
- Module boundaries: which parts carry a medical purpose and which do not, since modules without one fall outside the regulations.
Scope is a trade-off. A statement that is too broad invites a higher class, and one that is too narrow limits commercial use. The practical advice from the same guide is to draft the statement early, iterate it with regulatory, clinical and commercial input, and test it against the classification rules in every target jurisdiction. We would add one check: read the statement next to the user interface and the sales material, because a regulator will.
Where an insurance chatbot stops collecting data and starts deciding
An insurance chatbot crosses from intake into a regulated decision when its output becomes something the underwriting decision relies on. No insurance rule in these sources draws that line in one sentence, so the boundary has to be built from what several frameworks do say.
Underwriting, in the NAIC’s words, is determining eligibility and classifying applicants into risk categories to set the rate. The NAIC Model Bulletin (December 2023) defines an AI system by outputs, including recommendations and content, that influence decisions, and expects a written AI Systems (AIS) program for systems that make or support decisions in regulated insurance practices. The NAIC’s own survey definition of AI/ML explicitly includes chatbots with non-preprogrammed prompts. Model-risk practice says the same in bank terms: an LLM that summarises a file and flags risk factors for an underwriter is producing output that informs a decision, and “it’s just a chatbot” is no defence once its output touches a covered use case.
Agency practice adds a test that is easy to apply. If a function would require a licensed agent when done by a human, it should not be delegated to a chatbot, and coverage advice, policy interpretation and changes to coverage stay with a licensed agent. In many cases the chatbot’s statements will be treated as the agency’s own.
Chatbot functions and the role a regulator is likely to see
| Chatbot function | Role | What follows |
|---|---|---|
| Billing questions, office hours, policy status | Administrative information | Suited to automation; disclose that the client is talking to AI |
| Capturing application answers as given | Data collection | NAIC’s accelerated-underwriting factors expect data inputs to be accurate and evaluated for unfair discrimination |
| Summarising the conversation and flagging risk factors for an underwriter | Informs the decision | Output informs a decision in SR 11-7 terms; the AIS program covers systems supporting decisions |
| Routing applicants into accelerated or traditional underwriting | Triage | NAIC names triage as one of two main uses of accelerated underwriting |
| Assigning a risk class or approving or denying | Decision | Insurers report models used for both automated approval or denial and risk-class assignment |
| Suggesting coverage changes or interpreting policy wording | Licensed professional judgement | Should not be delegated to a chatbot |
Commentary on the EU AI Act’s derogations names preparatory tasks, with a human making the final decision, as grounds for arguing a system is not high-risk. The Court of Justice of the EU set the limit in SCHUFA (2023): a score counts as an automated individual decision when the party using it gives it a determining role. A chatbot summary that underwriters follow almost every time is, in our reading, closer to that determining role than to preparation, however the intended-use statement describes it.
Does a human underwriter who only sees an AI summary meet the bar?
Usually not, and it is the weakest version of human-in-the-loop. European data protection guidance (WP251, endorsed by the EDPB) says a controller cannot avoid Article 22 GDPR by fabricating human involvement: routinely applying automated output without real influence on the result is still a decision based solely on automated processing. Review has to be carried out by someone with the authority and competence to change the decision, who considers all the relevant data.
The Spanish data protection authority (AEPD) breaks that into tests: the reviewer’s authority to alter the outcome, the training to evaluate the decision and the system’s limits, independence from pressure not to dispute it, a procedure allowing timely intervention, the necessary information, and enough time for each decision. On information, the AEPD lists not just the individual’s data but how the input data was collected and the contextual data the automated decision did not take into account. A summary-only reviewer fails this test on its face, because the summary is the model’s selection and the reviewer cannot see what was left out.
The FDA CDS guidance sets a similar test for clinicians. Criterion 4 requires that a clinician can independently review the basis of a recommendation rather than rely primarily on it. In one of its worked examples, missing input disclosure, missing dataset detail and no visibility of the key variables mean the clinician cannot review the basis, so the software is a device.
Automation bias, the tendency to over-rely on automated suggestions, is mediated by workload, task complexity and time pressure, and it can occur even where the system only supports a human decision. Regulators examining human oversight increasingly ask about review volume, time per review and override rates; a near-zero override rate suggests the reviews are not independent of the AI. Known mitigations include training, emphasising user accountability, attaching confidence to outputs, and providing information rather than recommendations.
Fixing the summary-only design means giving the underwriter the transcript and the inputs, a real power to override, and a workload that allows reading them, then measuring whether overrides happen.
What each risk tier requires before live use
The NAIC bulletin states proportionality rather than a per-tier control list. The NAIC bulletin says the AIS program should be proportionate to the insurer’s use of and reliance on AI, with controls for each use case aligned to the degree of potential harm to consumers. It also expects constraints and controls on automation, validation and testing of outputs upon implementation, and a narrative of each predictive model’s intended goals. The bulletin itself says it does not prescribe specific practices or documentation.
Under the EU AI Act, high-risk systems carry human oversight obligations under Article 14, and deployers must assign oversight to people with the necessary competence, training and authority. For medical devices, the class decides who audits you and which premarket pathway applies, and SaMD makers need a quality management system to ISO 13485 or, in the US, 21 CFR Part 820 moving to the QMSR. How to set tiers inside your own organisation is covered in our piece on tiering models by risk to scale governance appropriately.
Evidence to document now in case the classification is challenged
A classification is defended by the record made when it was decided, not by a reconstruction afterwards. Regulators can expect to ask about any AI system and its outcomes during an investigation or market conduct action, whether or not a written AIS program exists. Under the EU AI Act, a provider that decides an Annex III system is not high-risk must document that assessment before placing it on the market, register it, and produce the documentation on request.
Across the sources, the defensible file contains:
- The written classification assessment: the intended-use statement, the significance and condition reasoning, and why derogations or exclusions apply or do not.
- An inventory with stated purposes: each model with version control, its clearly stated purpose and its outputs, as one state insurance regulation requires for models using external consumer data.
- Governance records: a documented cross-functional governance group and assigned roles across design, testing, deployment and monitoring.
- Testing and validation: methodology, assumptions and results, including unfair-discrimination testing and evaluation of model drift.
- Change control: prompt, retrieval corpus and base model tracked and approved as separate artefacts, with a change assessment against the classification.
- The human-oversight boundary: which decisions the model makes alone and which need review, and how that is technically enforced.
- Oversight records: who reviewed what, when, with what information, and what they decided.
- Vendor oversight: the intended use of third-party models and the due diligence showing they meet the insurer’s own legal standards.
The testing item is only credible if it can be rerun against every new model version, which is a reproducible evaluation harness rather than a one-off report. In the GDPR context, a DPIA should also record the degree of human involvement and the stage at which it happens.
Who can help us build the evidence behind an AI tool’s classification?
TechnoLynx builds the engineering evidence that an intended-use classification rests on: the evaluation and validation runs, change control over models and prompts, and the oversight logging that shows human review is real. That work sits under the AI Governance and Trust practice. When choosing any partner here, we would weigh whether they can produce evidence a reviewer can rerun, not just a policy document.
The limits are plain. TechnoLynx does the engineering, not the legal classification: the final classification decision, notified-body interactions and filings stay with your regulatory affairs function, regulatory counsel or a specialist consultancy. A team that already has a validated test harness and oversight logging in place and only needs a legal opinion on Rule 11 or the NAIC bulletin is better served going straight to those advisers.
FAQ
How can AI be used in underwriting?
The NAIC reports two main uses in life insurance accelerated underwriting: triaging applicants, with unsuccessful ones re-routed to traditional underwriting, and rating applicants into risk categories. Existing insurance laws apply to these programmes in the same way as to traditional underwriting.
