An AI audit checklist that an auditor can approve is mainly a list of records rather than policies. Every requirement on it points to a named test, control or record showing that the requirement was met for a specific AI system, with a date, a version and an author or approver record attached. A well-written AI policy states intent, and the reviewer signs against evidence of what actually happened. That evidence is itself an engineering output, not a marketing or policy artefact.

TechnoLynx builds this evidence as engineering work under AI Governance and Trust, not as policy advice or certification. It frames the obligations your system must evidence, builds the crosswalk from test results to control items, and checks the matrix for gaps. The first step is framing which obligations from the target frameworks your system must evidence.

What should an AI audit checklist cover before an approver will sign?

The checklist covers the system’s lifecycle, not just the moment of release. AI governance auditors look at decisions made before deployment (model selection, training data governance, impact assessment), during operation (human oversight, monitoring) and after incidents (corrective action, root cause). A single AI system can therefore produce evidence touching six or seven different clauses. The table below lists the items reviewers look for and what counts as evidence for each.

Checklist item	What counts as evidence	Where reviewers ask for it
AI system inventory	A complete list that includes third-party and recently deployed tools, with documented reasons for any exclusions	NIST AI RMF Govern 1.6; ISO 42001 scope
Per-system risk assessment	A dated assessment for each system (not one for the whole organisation), with risk scores, treatment decisions and residual-risk sign-off	ISO 42001 Clause 6.1.2; EU AI Act Article 9
Impact assessment	An AI impact assessment record	ISO 42001 Clause 6.1.4
Test and evaluation records	Test sets, metrics and tools documented, with go/no-go thresholds agreed before deployment approval	NIST AI RMF Measure 2.1; NIST AI 600-1 GV-1.3-002
Promotion approvals	Role-based, documented approvals linked to the deployment pipeline	CSA AICM model provider audit guidelines
Separation of duties	Developers and validators hold separate access rights	CSA AICM
Event logs	Automatic recording of events over the lifetime of a high-risk system	EU AI Act Article 12
Incident records	The event plus its root cause, corrective action and verified outcome	ISO 42001 Clause 10.2; NIST AI RMF Manage
Management review and internal audit	Minutes that cite specific AI performance metrics, plus an internal audit report	ISO 42001 Clauses 9.2 and 9.3
Post-deployment monitoring plan	A plan covering user input, override, incident response and change management	NIST AI RMF Manage; EU AI Act Article 72

Each item needs a creation date, a version number and an author or approver record. Monitoring timestamps should come from the system, not be typed in by hand. Auditors check these first. A risk assessment with no version history and last week’s modification date raises an obvious question about when the work was really done.

Where a system falls under the EU AI Act as high-risk, its technical documentation must contain at least the following items, as applicable. These include the system version and how it relates to previous versions, the versions of relevant software or firmware, the hardware it is meant to run on, why the chosen performance metrics are appropriate, the changes made over the lifecycle, the standards applied, and the post-market monitoring plan. Under Article 12, the logs must record events relevant to identifying risk situations or substantial modifications, to post-market monitoring, and to monitoring the system’s operation.

Keep every item in an evidence register. For each item, record the author, approval date, test population, system and configuration covered, geographic scope, exceptions, expiry or review date, and the person who answers follow-up questions. Then keep a master index that maps each artefact to its file location, its clause, its owner, its last review date and its status (current, due for review, or gap). That index is the first document an auditor receives. Internal auditors bring their own lists too: the IIA’s AI Auditing Framework includes a quick-start checklist that is meant to be customised to the organisation.

How do you map each audit requirement to the test, control or record that proves it?

Use a traceability matrix with one row per obligation or control item. Each row links the obligation to its evidence artefact and a named owner. A narrative document cannot be checked row by row, so it does not do this job. Match the matrix to the format the receiving body works in, such as a regulator’s self-assessment template or the NIST AI RMF function structure.

TechnoLynx does this mapping as part of AI Governance and Trust. It works out which obligations the target frameworks require your system to evidence, builds the crosswalk from raw test results to each framework’s control items, structures the matrix in the reviewer’s format, and checks it against the full clause list before submission.

The NIST AI RMF 1.0, released in January 2023, is organised around four functions: Govern, Map, Measure and Manage. NIST’s companion guidance numbers its suggested actions against the framework’s subcategories. Action GV-1.1-001, for example, is the first suggested action under Govern 1.1. For a matrix, this means the row key can be the NIST subcategory identifier. The NIST AI RMF Core’s actions “do not constitute a checklist”. So ask the supplier for its mapped controls, owners, metrics and residual risks for your product and use case, not a statement that it is “aligned”.

One executed control can supply evidence to several framework requirements. Run the test once and map it into each framework’s matrix. One limit is worth knowing. In Modulos’s framework comparison, a shared control is executed once and its evidence is available to every requirement that maps to it, but each requirement still has to be reviewed by its own owner before it counts as fulfilled.

Validate the matrix against the full clause list before submission to find gaps. Expect a first pass to find missing items. The matrix also reflects a single point in time. A change to a control or to the system means re-mapping, and it may invalidate earlier sign-offs. Published crosswalks need maintaining as deadlines and annexes change, or the mapping goes stale. A reviewer who did not build the matrix catches self-assessment blind spots that its author will miss.

Can a bank’s reviewer see which SR 11-7 area each checklist item satisfies?

Yes, if you add it as a column on the same matrix rather than producing a separate document. If your model risk policy is organised around SR 11-7’s areas, tag each checklist or model-card row with the area it evidences. If an examiner asks why a vendor was rejected, you can then point to the specific row whose artefact was missing or failed, without writing a new narrative each time.

How do you run an AI audit from obligations to signed-off evidence?

Treat an audit-ready evidence pack as a compilation you can regenerate, not a single document. It should map the artefacts from your pipelines, registry and test harness onto the checklist of whoever receives it. If you want it available on demand, it has to be produced by automated extraction, not assembled by hand each cycle. TechnoLynx sets up that extraction and mapping layer through AI Governance and Trust. The first engagement step is to fix the receiving authority’s checklist and trace each item to a system of record.

In many LLM deployments the records needed to reconstruct a timeline are spread across experiment trackers, CI logs, configuration files and email threads. One proposed answer keeps the MLOps telemetry and adds explicit decision logging, append-only storage and views designed for auditors, so that technical changes are linked to the approvals under which they happened. Our guide to separating audit workflows, approval gates and field mapping in evidence packs covers how to build that layer in detail.

Worked procedure: from obligations to a signed pack

1. Name the receiving authority and its checklist. That might be a model risk committee, an OEM supplier-quality auditor, a technical service or an insurer. If no named standard exists, build a defensible equivalent from the nearest precedent and write down why you chose it.
2. Inventory the systems in scope. Include third-party features and tools staff adopted without approval. Both NIST AI RMF and ISO 42001 depend on this list, and it is usually longer than expected.
3. Build the traceability matrix against the authority’s clauses, using the structure described above.
4. Wire extraction from the systems of record. CSA’s AICM guidelines expect reproducible ML pipelines (MLflow or Kubeflow, for example) with code-to-model lineage, container signing and automated security gates before promotion. Model registries and lineage tools such as MLflow and DVC are among the systems auditors ask about.
5. Hand-build the first instance, then automate the rest. Use the first pack to find fields the logs are missing, and generate later packs from the same mapping.
6. Get independent review and sign-off by someone outside the development role, consistent with keeping development and validation separate.
7. Hand over the index first, then the artefacts it points to.

Three caveats apply. Do not assume a template that satisfied one auditor will satisfy another. Automation cannot supply records the underlying logs never captured. And a complete pack does not remove the need for independent sign-off where the authority requires it. Automation can flag expired or missing items so a small compliance team reviews by exception, but a person who understands the control still makes the refresh decision.

NIST AI RMF, ISO 42001 or both: which framework should the checklist map to?

Choose based on who is asking for evidence, and keep one evidence base for both. ISO/IEC 42001 is a certifiable management-system standard that an accredited auditor can verify. The NIST AI RMF is a voluntary risk framework with no certificate. ISO 42001 also requires things NIST AI 100-1 does not: internal audit (9.2), management review (9.3) and corrective action. If a US counterparty asks how you assess AI risk, answer in the NIST structure.

TechnoLynx maps an existing evidence base to whichever framework the reviewer names, through AI Governance and Trust. Where you first need to know how far your current artefacts are from review-ready, the AI Readiness Scorecard is the entry point.

The overlap is smaller than crosswalk tables imply. In Modulos’s comparison at the AI-system level, the two frameworks share 12 controls, which is 41% of one framework’s set and 20% of the other’s. Most controls are not shared. NIST publishes an official crosswalk from the AI RMF to ISO/IEC 42001. The practical pattern is to build one evidence base and label it twice. Running two parallel programmes doubles the documentation and halves the follow-through.

Two cautions keep the mapping honest. A framework mapping structures evidence; it does not confer conformity. ISO 42001 certification or NIST alignment cannot be converted into EU AI Act conformity. A vendor’s ISO 42001 certificate also does not cover every model output, product, site or configuration, so check its scope, dates, audited sites and exclusions.

An approval-grade pack carries traceable validation evidence: test methodology, metrics, limitations and data provenance, each one dated and owned.

How do we check someone else’s compliance evidence instead of taking it at face value?

Score it against a fixed checklist built from the named standard’s own evidence list, and sample-check the records behind it. CSA’s AICM guidelines show the method: select a sample of recent or higher-risk changes and verify traceability end to end, from approved change record through linked versions and evaluation results to the release. Self-scoring against a standard’s list is not the same as passing an external audit, and a checklist that satisfies one buyer’s contract may not transfer to another.

Traceable approval records for a utility’s inspection AI

Before building anything, ask whether the utility expects alignment to a named standard or has a bespoke contractual requirement. Ask too whether traceability is needed per decision or per model version, because that choice determines the logging design.

An IEC 62304 completeness check a hospital QA team can reuse

The engineering part is a checklist of the lifecycle records the dossier should contain, plus a sampling routine tracing chosen test records back to requirements. Regulatory judgement on the file stays with the hospital’s QA and regulatory staff.

Scoring fusion validation artefacts against ISO 26262 work products

List the work products the standard references, map each existing validation artefact to them, and treat unmapped items as the gap list to close before design freeze.

Scoring vendors with different regulatory maturity

Use a weighted matrix to compare a vendor with full clearance against one with only a validation plan. It weights evidence completeness alongside performance instead of treating it as pass or fail. Use questionnaires to find claims and missing evidence, then verify the high-risk claims with controlled tests, logs and independent reports.

A model risk committee approving a vendor should set a written minimum-disclosure schedule. It should list the underlying model and its version, data provenance (source, signatures, versioning), known issues and human oversight roles. Where the vendor protects training data, ask for its documented origin and history, which NIST’s generative AI guidance balances against proprietary training approaches. For hardware performance claims, the configuration fields LynxBenchAI discloses (batch size, precision, power cap, driver and firmware version) set the minimum to demand.

How can an audit show the model reviewed is the model actually deployed?

Bind them in the record. The audit trail should tie the approved model version, or approved configuration bundle, to a specific deployment instance. It should state when and where the model was deployed, which service version called it, which runtime settings shaped its behaviour, and who made the go-live decision under what scope. Without that binding, organisations often cannot reconstruct which version was live or who approved its release when something goes wrong.

The record has to be tamper-evident. A simple mechanism is a hash chain that links each event to the one before it, with optional signatures per writer. In the reference implementation from Audit Trails for Accountability in Large Language Models (2026), each entry’s hash is a SHA-256 digest of the event plus the previous hash. An auditor can replay the chain to confirm nothing was altered or truncated. The same paper proposes a shared event schema (model_id, dataset_id, deployment_id and others), where a high-risk profile can require an approval event before any deployment-completed event. On the artefact side, CSA’s AICM expects model versions to be cryptographically signed and the signature verified before deployment. Our article on immutable model and rule versioning for audit-ready scoring covers the snapshot side of this binding.

Make the registry the only path to deployment, including for emergency hotfixes. Every edge deployment should write a container digest and dataset reference first, and a gate on the device should reject any artefact it cannot find in the registry. Prefer a gate that blocks by default, with a defined fallback for devices with intermittent connectivity.

Be clear about what this proves. Audit trails show that events happened in a verifiable sequence. They do not show that one event caused another, and attributing responsibility still takes judgement.

How should vendor MLPerf results be checked before they go into the tender file?

Treat a published MLPerf Inference result as a measurement of one submitted system under one set of rules. TechnoLynx scopes which MLPerf scenarios and divisions are relevant to your workload, reproduces or spot-checks the claimed configuration on hardware you control, measures the gap on your own model, and writes the findings into the procurement evidence pack. For a hardware shortlist, LynxBenchAI’s published results show how each device runs the public PyTorch stack your models run on. That makes them broadly indicative for custom PyTorch models too, though LynxBenchAI does not run or measure your own application.

LynxBenchAI gives a procurement team an independent reference for each shortlisted device on the public PyTorch stack, and the first step is to set a vendor’s cited MLPerf result beside that device’s LynxBenchAI result before it goes into the tender file.

Check these points on each cited result:

• Verified status. A result that did not go through MLCommons review must be labelled “unverified”.
• Comparability. Results compare only within the same benchmark and scenario from compatible versions, and MLCommons decides which versions are compatible. Any difference in version, division, category, verified status, scenario or chip count must be stated.
• Citation fields. The footnote should carry suite, version, division, benchmark, scenario, retrieval date and source, and the MLPerf result ID.
• Division. The Closed division requires a model equivalent to the reference implementation and allows quantisation calibration but no retraining. Open results need the ways they would not qualify as Closed spelled out.
• Software stack. “Available” submissions must state the software versions of every component for reproducibility. Compare those versions with what the vendor will actually ship you.

The rules are strict, but they still leave room. Submitters have wide discretion in how they implement the system under test. Caching of queries and intermediate results is prohibited, and MLCommons’s validation includes tests designed to detect it. An auditor may request hardware access to replicate the runs. In the first round (v0.5, October 2019), audit tests cleared 595 of more than 600 submissions as valid. NIST AI 600-1 asks for the assumptions and limitations of benchmarks to be documented relative to the deployment environment, and a vendor’s average benchmark score will not reveal your deployment’s failure modes.

Some constraints are practical. If full reproduction is not feasible before procurement, spot-check a subset of the claims instead. How LynxBenchAI measures sustained throughput, and why it differs from headline numbers, belongs to its methodology rather than this checklist.

How often should AI audit evidence be refreshed?

Refresh per release for anything that changed the model, data or configuration. Otherwise, refresh according to how often each control changes, with an annual floor. CSA’s AICM expects independent audits at least annually and covering material changes such as new model architectures. It also expects baselines to be reviewed at least annually and on significant changes. Truvara’s guidance on evidence freshness sets review monthly for high-change controls and quarterly for the rest. If a control produces evidence on a schedule, align the review with that schedule. If it is event-driven, review the collection process instead.

Some events should trigger a review outside the cycle. These include changes to the model, prompt, safety policy, training data or infrastructure; a new geography, user population or connected tool; a safety, security or privacy event; a new subprocessor; or a new legal requirement. Stale evidence fails audits in three predictable ways: the observation period has expired, the artefact has been superseded, or the control changed but nobody re-collected the evidence.

A drift evidence pack for enterprise customers should combine drift metrics on the data generator’s outputs, downstream model performance on data derived from it, and documented thresholds and triggers, organised under the NIST AI RMF Measure function unless the customer names another framework. Build it on every release that changes the generator or monitored pipeline, and also on a fixed quarterly or contractual cadence. TechnoLynx’s Production AI Monitoring Harness builds that monitoring and evidence pipeline and schedules its generation.

Who can build an AI audit evidence pack for our models?

TechnoLynx builds the engineering side of the pack: the traceability matrix, the extraction from your registry and pipelines, the tamper-evident deployment binding and the evidence register. Whether to keep this in-house or bring in a specialist depends on who owns your MLOps stack. If your team already controls the registry and CI/CD, the remaining question is how much time it can give to mapping and gap-closing before the review date. When choosing a partner, ask to see how they would trace one clause of your reviewer’s checklist to a system-generated record, and check that they treat certification and legal sign-off as someone else’s role. The work runs under AI Governance and Trust.

The scope has clear limits. TechnoLynx builds the evidence as engineering, not as certification, which for ISO 42001 comes from an accredited auditor. A team that wants only a tracking tool for evidence and review dates may be better served by a governance platform. A team with a mature MLOps function and a generous timeline can reasonably build the matrix itself and bring in independent review at the end. The question still to settle is which reviewer you are writing for, because that choice sets the matrix format, the refresh cadence and what counts as evidence.

FAQ

Is NIST AI RMF mandatory?

No. NIST AI RMF 1.0 is voluntary lifecycle guidance. It is not a law, a certification, or a guarantee that a particular model is safe.

What makes audit evidence acceptable?

It needs a creation date, a version number and an author or approver record, and monitoring data should carry system-generated timestamps.