A model risk audit checks whether your model risk management framework works the way your approved policies and procedures say it does. Auditors look at governance, policies, the model inventory, change controls, validation standards and how findings are tracked. Internal audit’s job is to judge whether those practices are rigorous and effective, not to validate the models again. So the package that survives the audit is not a single document. We would build it as one versioned evidence base (development, validation, approval, monitoring), indexed separately for the external auditor and the prudential supervisor, and signed so an outsider can check what was true on any past date.

That evidence is an engineering output. Model-risk committees, auditors and compliance owners must be able to read, defend and sign against it, so it is not a marketing or policy artefact.

What does a model risk audit examine, and how is internal audit different from validation?

The Institute of Internal Auditors (IIA) describes internal audit’s role in model risk management as assessing the effectiveness of the whole framework: governance, policies, procedures and the activities that address model error. Internal audit also checks whether model output is used in ways that fit the model’s stated purpose. Internal audit is the third line of defence. It reviews whether the framework is sufficient, gives an opinion on the overall process and reports directly to the board or its delegates. The Basel Committee adds that internal audit should not be involved in designing, selecting, implementing or operating specific internal controls.

Validation is a different activity. In the IIA’s wording it verifies whether a model functions as intended. An independent party usually performs it, separately from the testing done by developers and users. A validator should have had no involvement in development and no financial considerations tied to the model, and should have enough authority to get errors corrected. Where a small organisation has to let developers run the validation tests, an independent party such as internal audit should review the results.

The revised US guidance, issued jointly by the Federal Reserve, the OCC and the FDIC as SR 26-2, which supersedes SR 11-7 and SR 21-8, states the principle behind both roles as “effective challenge”: critical analysis by objective experts with the expertise, independence and organisational standing to force change. In practice this decides what the documentation has to show. So we would have each artefact say who did the work, in which role and when, and not only what the metrics were.

What the model inventory has to show

Under OSFI Guideline E-23, the model inventory is the institution’s system of record for key model information. It contains every model whose inherent risk is non-negligible. It should be kept at enterprise level, accurate and evergreen, and updated promptly when a model is modified or its use, risk rating or performance status changes. It should also keep decommissioned models for a reasonable period. E-23 separates the model owner, who coordinates development, deployment and monitoring and maintains the documentation, from the model approver, the unit, individual or committee with authority to approve use. E-23 takes effect for all federally regulated financial institutions on 1 May 2027.

The PRA’s supervisory statement SS1/23 expects the inventory to cover models in use, models in development and decommissioned models. A GARP guide to SS1/23 (2023) notes that the inventory must now record purpose, use, outputs, significant changes, post-model adjustments, and actual against intended use. Taken together, the fields an auditor will look for in each entry are:

• Model owner, developer and users, documented for each model.
• Purpose, intended use and any restrictions on use.
• Inputs, upstream models and outputs, with their intended use.
• When the model was last updated, and significant changes made to it.
• The current risk tier.
• Dates of completed and planned validation activity, and exceptions to policy.
• Known limitations, post-model adjustments and the reasons for them.
• Open issues and the status of their remediation.

The IIA guide expects internal audit to confirm that the inventory exists, that controls keep it complete and accurate, and that its level of detail fits the complexity of the models.

Recording the risk tier, who set it, and how deep each tier goes

E-23 says the inherent model risk rating should drive the frequency, intensity and scope of review. It also drives the documentation requirements, the level of authority needed to approve the model, the depth of monitoring and the interval at which the rating itself is reassessed. SS1/23 asks policies to set out the tiering approach, the data sources behind it, who assesses materiality and complexity, how often tiering is re-performed and how tier assignments are approved. Validators should reassess individual tier assignments during validation and revalidation. The tiering approach as a whole should be periodically reviewed, including whether tiers are recorded accurately in the inventory.

The revised US guidance says model purpose and exposure together determine materiality. For a model judged immaterial, model risk management may amount to identifying it and watching for conditions that would make it material. E-23 says externally developed models, including a parent organisation’s, need their own standalone rating, because the parent’s rating may not reflect the risk to the subsidiary. It also warns that a model used outside a production environment may deserve a higher rating, because production controls no longer mitigate its risk.

We would record the tier as a decision rather than a field value: the inputs used, who assessed them, who approved the tier, the date, and the policy version it was assessed against. Without that record, a later reassessment cannot show what changed or why.

External auditor versus supervisor: what each expects from the same model documentation

The workable package is one evidence base with two indexes laid over it. SS1/23 states that its principles complement existing supervisory expectations for particular model types and do not supersede them. The Basel Committee’s 2014 guidance on external audits frames the auditor’s task as identifying and responding to the risk of material misstatement in the financial statements.

The NAIC Model Bulletin (2023) says insurers should maintain a written AI Systems program addressing governance, risk management controls and internal audit, including oversight of predictive-model inventories and documentation.

The two audiences meet at defined points. The PRA expects firms to give their audit committee a report on the effectiveness of model risk management for financial reporting at least annually. That report should also reach the external auditor in time to inform its assessment of material misstatement risk. The external auditor in turn has a duty to report matters of material significance to the supervisor.

Cross-index of one evidence base to two audiences

Evidence in the base	What the external auditor uses it for	What the supervisor uses it for
Governance, policies, roles	Assessing the control environment and the key control functions, including risk management and internal audit	Board-approved policies and clearly delineated responsibilities across the lifecycle
Data sources and quality tests	The information system relevant to financial reporting is a component of internal control	Documentation of data sources, proxies and quality test results
Validation evidence	The model risk management effectiveness report informs the misstatement risk assessment	Independent review and periodic revalidation feeding approval
Management judgement and adjustments	Professional scepticism means challenging management’s assertions and documenting the conclusions	Material post-model adjustments documented, approved, and reported with and without the adjustment
Disclosures	Evaluating whether disclosures match the bank’s risk profile and control framework	Largely auditor-specific
Ongoing monitoring and limitations	Monitoring of controls is one component of internal control	Whether the model still performs as expected as products, data and markets change

The overlap is largest in governance, data and validation evidence. The two views diverge on disclosure (an auditor concern) and on monitoring and limitations (a supervisor concern). Mapping a draft against both indexes shows its gaps quickly. Any item that appears in only one column, or that is weakly evidenced in either, is the work still to do. The mapping has to be tailored to the jurisdiction and the regulator. It organises evidence for review and does not replace sign-off by either party.

The evidence base belongs in a repository where every artefact carries a control ID, framework mapping, owner, effective date, version and source. Access should be least-privilege, with immutable logs of who viewed, changed or downloaded each item. We would give reviewers a scoped, read-only export or permissioned view instead of the codebase. A manifest of cryptographic hashes lets them confirm that nothing changed after the export was sealed. TechnoLynx sets up that repository structure, the export approach and the restricted reviewer accounts, and tests that reviewer access cannot reach the wider codebase or leak its history.

A good test of the package is whether someone other than the original analyst can reproduce the model’s output from it. SR 11-7 set the bar as documentation that lets parties unfamiliar with a model understand how it operates, its limitations and its key assumptions. TechnoLynx builds that versioned, reproducible methodology record as part of its AI Governance and Trust work, and the reproduction test is worth running well before a handover, because tacit judgement calls only surface when someone else tries.

How do you prove the approval gate was in force at every past promotion?

You prove it with records made at the time, not with today’s pipeline configuration. Design output is approved before release, and the approval records the date and the signature of the approver. Design changes are reviewed and approved before they are implemented. E-23’s cover letter leaves institutions to decide at which stages approvals are needed, but says approvals should come before a model change is implemented and after periodic reviews. The IIA’s audit programme tests this. It asks whether changes were logged, reviewed and approved, and whether every model in the inventory was validated before implementation, or was formally approved for use and then validated promptly. Where an urgent business need requires use before validation, the revised US guidance describes sound practice as closer attention to limitations and controls such as limits on use.

A gate definition in today’s repository says nothing about last March. To reconstruct a past promotion, we would join these items for that date:

1. The model version or artefact hash that was promoted, and when.
2. The approval record for that version, with approver identity, authority and date.
3. The version of the gate or policy in force at that moment, from its own change history.
4. The validation report the approval relied on, linked by version.
5. A manifest with hashes and trusted timestamps that shows these records were not edited afterwards.

Missing release gates on regulated workflows are among the production AI failures that release-readiness reviews, an engineering artefact, are meant to catch. The tamper-evidence design of the audit trail itself is a separate topic. The design of record schemas and promotion gates is covered in standardizing model documentation and approval: schemas, gates and go/no-go decisions.

What counts as a material model change, including changes on the vendor’s side?

E-23 lists the events that should prompt a model review. They are new models, modifications to algorithms, parameters or supporting operational components, breaches of performance thresholds, significant data changes, and scheduled risk-based reviews. For self-learning models, E-23 leaves institutions to set their own internal criteria for when the model has materially changed. SS1/23 requires the policy to state the materiality criteria used to judge the impact of a prospective change.

Under the EBA rules for IRB and IMA internal capital models, material changes need permission from the competent authority. Other changes need either an ex-ante notification at least two months before implementation or an ex-post notification. The thresholds are measured in risk-weighted exposure amounts. A decrease of 15% or more in risk-weighted exposure amounts marks a change as material. A non-material change that decreases risk-weighted exposure amounts by at least 5% needs ex-ante notification. All other changes are notified after implementation, at least annually. Some changes count as material whatever their numerical impact, such as a fundamental model re-design or a switch from a simple linear model to a random forest or neural network. Similar modifications are bundled into one change, so that splitting them cannot dilute the threshold.

We would carry the same pattern to other models: a numerical threshold, a short list of changes that are always material, and a bundling rule. FDA’s guidance on 510(k) device changes has its own change assessment. It uses a risk-based assessment of each change, separately and in aggregate, and the decision is documented whether or not a submission follows.

Vendor models fall within model risk management too. SR 26-2 describes as sound practice understanding a vendor model’s design, development data and performance, monitoring its outcomes on an ongoing basis, and documenting any customisation as part of validation. The IIA guide adds that the organisation should obtain the vendor’s own monitoring results and details of any modifications, and should keep a contingency plan in case the vendor can no longer support the model.

For a vendor model, we would put the re-approval trigger both in the contract and in the annual approval conditions. It should be a binary, named event, such as a base-model version change, a change of model provider or a change to inference data retention, never “any significant change”. Each firing is recorded with its date, how it was detected, the evaluation and the outcome, even when the change is judged non-material. TechnoLynx validates it by simulating notifications and adds monitoring for missed or late ones.

Does the approval record have to cover PHI handling and a BAA before validation uses patient data?

Yes, where the validation uses real PHI. A vendor that creates, receives, maintains or transmits PHI on a covered entity’s behalf is a business associate, and the relationship requires a written agreement before any PHI flows. We would keep clinical validation approval and PHI approval as separate requirements, neither substituting for the other.

That applies to evaluations too. One BAA guide for AI vendors advises that where the vendor would act as a business associate, no PHI goes to it without the agreement and any evaluation is limited to synthetic data or data that privacy or legal reviewers have confirmed is de-identified. The agreement has to hold at every stage the data passes through: prompts and uploads, inference, storage, embeddings and other derived data, audit logs, and deletion at termination. So we would have the approval record document where and how the vendor’s model processes PHI. It names each subprocessor, its written retention terms and its own downstream agreement. It treats vectors derived from PHI as PHI unless a documented de-identification determination says otherwise. It treats any log plane that captures PHI as a PHI system with its own safeguards and retention schedule.

The HHS model BAA lets the covered entity request the business associate’s most recent independent HIPAA compliance report, HITRUST certification or an agreed third-party audit report. We would keep that report in the evidence base next to the signed agreement. Even so, a signed BAA is necessary but is not the whole vendor review. The contract has to be read beside the architecture, configuration and security evidence. The clinical side still needs its own validation, showing the device performs its intended use safely and effectively for its users.

The decision logic is short. If real PHI will be used in validation, the BAA has to be executed before that data flow begins. If only synthetic or confirmed de-identified data is used, evaluation may proceed while counsel and compliance review the proposed use. Institutional policy can add its own residency requirements. Policies, BAAs and related documentation fall under HIPAA’s minimum six-year retention.

To prove PHI-bearing inference logs stay in the contracted US region, we would put three things side by side: configuration evidence enforcing the region, an external attestation scoped to cover it, and the approval record showing residency was reviewed. The approval record alone shows that review happened, not that today’s configuration enforces anything. A residency commitment belongs in the agreement itself when institutional policy requires one.

Out-of-scope asset classes: wording the exclusion so nobody reads it as “probably fine”

We would write the intended-use boundary as an enforceable control, not a description. Name every asset class absent from or underrepresented in the training data. State that use outside the list is prohibited, not merely discouraged. Back the statement with a technical control. Route any extension through re-validation and re-approval. E-23’s documentation standards include limitations and restrictions on use. SS1/23 makes model users accountable for using a model consistently with its intended purpose and for taking its known limitations into account. SS1/23 also expects the independent review to critically analyse whether development data is relevant and complete for the portfolios, products and assets the model is meant to serve. We would list the gaps by asset class in that review.

Careful wording stops “out of scope” drifting towards “untested but probably fine”. A formulation that resists the drift is: “Not validated for [asset class]. Use for [asset class] is prohibited until re-validation and approval under the model risk policy.” This puts the reason (no validation) and the consequence (prohibition) in one sentence. We would not rely on a policy statement on its own. The control should therefore be as concrete as the inputs allow: an allow-list on asset-class codes at the input boundary, rejection or flagging of excluded records, and a monitoring alert on attempted use.

We would treat extending scope later as a formal change, not a reinterpretation. One AI governance guide lists scope expansion as a standard re-assessment trigger. SR 11-7 expected ongoing monitoring to verify that any extension beyond the original scope is valid. Any temporary exception should be reported, supported by senior management and subject to post-model adjustments. We would record the list’s dependence on documented training-data provenance, and any gaps in the input controls, as known limitations.

Will the change-control trail satisfy Quality, the auditor, and a regulator whose focus shifts?

It has the best chance when the documentation is built on a recognised framework rather than a bespoke format, and when it records intended use, validation evidence, separate approver, executor and reviewer roles, and the change history. No one can promise in advance that a regulator will accept a particular shadow-mode validation approach.

TechnoLynx frames an AI-assisted inspection change against the applicable CSV and model-risk documentation requirements. TechnoLynx defines the shadow-mode validation and an evidence pack with approve, execute and review roles mapped, runs the validation on the target inspection stack, and packages the result for internal QA and external audit. An AI Readiness Scorecard is where that engagement starts.

CSV change control for an AI-assisted inspection change

The device rules require procedures to identify, document, validate or verify, review and approve design changes before implementation. Changes and approvals are recorded whether or not a new premarket submission is needed. A process whose results cannot be fully verified by later inspection must be validated, and changes to it need review and revalidation where appropriate. Since 2 February 2026, Part 820 has been FDA’s Quality Management System Regulation, which incorporates ISO 13485:2016 by reference; the clauses above are the earlier Quality System text. For an AI model specifically, FDA’s AI device guidance asks for the version history, including any differences between the tested and the released model and their possible effect on safety and effectiveness.

We would show role separation explicitly in the record. The validator should have had no part in development. SR 26-2 also flags conflicts of interest between development and validation reporting lines as a governance concern.

Pricing-model governance when the regulator’s review focus moves

For insurance pricing, the NAIC bulletin expects the AI Systems program to cover the whole lifecycle, from design through monitoring, updating and retirement, with controls proportionate to the potential harm to consumers. For a specific model under investigation or examination, the bulletin lists documentation including data lineage, bias analysis, thresholds, validation and model drift evaluation. We would align the trail to an established model risk framework rather than a home-grown scheme.

Frameworks change too. SR 11-7 was replaced after fifteen years of supervisory experience. So we would have each approval record name the guidance, and its version, that the approval was assessed against. A later reviewer can then see what the standard was at the time, rather than judge an old decision against a new text.

The board diagram and the on-prem hosting selection record

These are two different artefacts. We would give the board and the auditors one consolidated chain-of-trust diagram with an evidence index for each compartmentalisation boundary. For IT security we would keep a selection record with the criteria, the comparative evidence scoped as such, the rationale, a named owner and a refresh trigger. A benchmark report is one input to that record, never the whole of it.

TechnoLynx catalogues the existing chain-of-trust and provisioning documentation and maps it onto one reference chain from silicon root of trust to signed firmware. TechnoLynx produces the consolidated diagram with an evidence index per boundary, reviews it against the applicable audit checklist, and sets up the process that keeps both current as the architecture changes.

One diagram for the board and the auditors

We would attach evidence to each boundary (key attestation records, signing logs, provisioning procedures) rather than present a diagram on its own. The diagram should stay traceable to the detailed documentation rather than replace it.

A selection record IT security will sign

SR 11-7 expected decision makers to document the information that led to selecting a model and its subsequent validation. The IIA audit programme checks that the decision to build or buy was documented. A benchmark report measures comparative performance on the workloads tested. It says nothing about model risk, security or compliance review, so it cannot stand in for the record. Ownership is best assigned by role rather than by name, so the record survives staff turnover. A model swap should be one of its refresh triggers.

The same discipline covers out-of-cycle revalidation. E-23 counts performance breaches and significant data changes among the events that should prompt a review. Each trigger event, including those judged non-material, is recorded with its date, how it was detected, who evaluated it and the outcome. TechnoLynx builds the approval-record schema and the generation workflow that binds each recalibration decision and memo to the exact model and parameter version it approved, so the trail regenerates with every retrain and can be reused at the next exam.

What the signed approval record needs before go-live, and who signs it

Before go-live, we would have the record name the model version, the intended use and its limits, the validation evidence, the risk assessment, the monitoring plan and the accountable approvers. Each approver signs and dates it. FDA’s device guidance and model risk management practice can be adapted into an internal template covering the same core fields.

TechnoLynx implements the record as a production-admission gate. The workflow blocks go-live until every required artefact and named signature is present, and TechnoLynx tests that it actually blocks, then keeps it running across releases as versions and re-approval triggers come through.

FDA’s draft guidance on AI-enabled device software functions, which contains nonbinding recommendations, sets out much of the content. It asks for a statement that AI is used, with the device inputs and outputs, the intended users and their training, and the intended workflow and degree of automation. It asks for performance metrics with confidence intervals and subgroup analysis, a risk management file, and a performance monitoring plan that covers drift, input data integrity and the deployment of corrective updates. The version history connects the record to the gate, because it shows which model was tested and which was released. The signature practice follows the design-control pattern: dated and signed before release.

On who signs, one published clinical AI governance committee charter sets out a minimum membership. The chief medical officer chairs, alongside the CMIO, clinical informatics, the chief compliance officer or a designee, the CISO or a designee, service-line representatives, a patient safety officer and at least one frontline clinician. That committee approves or rejects deployments and material updates. Your own committee charter, not the approval record, decides who must sign. The record should cite the charter section it relies on.

A client-facing commentary generator that neither prices nor decides still needs an explicit home in the policy. The NAIC bulletin’s definition of an AI system includes generated content that influences decisions. SR 26-2’s model definition centres on quantitative estimates and excludes deterministic rule-based software. So whether the generator counts as a “model” under bank policy is a definitional choice the policy has to make on purpose, with a named sign-off path through model risk and compliance, and retained validation, approval and monitoring evidence from the pilot onwards.

Showing auditors that monitoring runs and past findings were closed

The revised US guidance describes ongoing monitoring as checking whether a model still performs as expected as products, exposures, clients, data relevance and market conditions change. It also covers procedures for responding to issues before and after approval. The IIA expects the framework to include processes for classifying, escalating and tracking validation findings. SS1/23 asks for significant deficiencies to be documented, reported to stakeholders and tracked to remediation in the model inventory. It also places responsibility on the approval authority for making sure validation recommendations are acted on.

Finding closure has its own trail. The Basel Committee expects internal audit to follow up with senior management on corrective measures. The head of internal audit reports to the board or audit committee the status of findings not yet rectified, and senior management is accountable for acting on them in time. The OCC handbook expects the audit committee to keep minutes and records of its meetings and decisions.

We would count a finding as closed only when an auditor can follow it from its ID to the change record or commit that fixed it, to the re-test evidence, to a verifier who is not the person who made the fix, with dates at each step. A ticket marked “done” does not meet that test. We would hold monitoring evidence to the same test. We would supply run logs with timestamps, the alerts raised and the decision on each one. A dashboard is not the deliverable. This page leaves drift detection design and retraining trigger policy aside. For this page, what matters is that their outputs land in the evidence base with the model version attached.

Who can build an audit-ready model documentation package for a regulated team?

TechnoLynx builds the engineering side of the package in its AI engineering for finance work: the versioned evidence base, the cross-indexes for auditor and supervisor, the approval gate that blocks promotion until named sign-offs exist, and the change triggers, including vendor-side ones, that force re-review. Whether to build in-house or bring in a specialist depends on whether your platform team can treat evidence generation as pipeline work alongside delivery. If it cannot, a specialist shortens the path. When choosing a partner, ask to see a sample of evidence regenerated for a past date, not a policy template.

The engagement has clear limits. It is engineering work, not policy advice or certification. Other routes are fair choices: an internal model validation function can extend its remit, audit firms’ advisory arms work on framework design, and regulatory counsel is the right route for BAA language and residency law.

Can you regenerate the package for any past date?

The real test of the package is not whether it looks complete this quarter. It is whether, for any past date, you can regenerate what was approved, by whom, under which policy version and on what evidence, without asking the people who were there. If that reconstruction needs a meeting, the gaps are in the pipeline and not in the prose. SR 11-7 was itself superseded after fifteen years. A team whose records already name the standard each approval was assessed against loses the least when that happens.