A heatmap that looks plausible on a damage photo will not, by itself, defend an automated severity score. Under a risk-based validation approach, explainability in insurance claims needs three things working together. The first is an overlay that has been tested for faithfulness to the model, not judged by how convincing it looks. The second is a documented standard saying what must be shown, and at what fidelity, for each tier of decision. The third is a record that lets an adjuster, a claimant or a regulator replay how the payout was reached. Some saliency methods produce maps that are independent of both the model and the data, so relying on visual assessment alone can mislead.
How deep each of the three goes should follow the stakes. The NAIC model bulletin asks insurers to scale their AI controls to the nature of the decision, the potential harm to consumers, human involvement in the final decision, the outcome’s transparency and explainability to the consumer, and reliance on third parties.
Can a claims AI justify its severity score with a heatmap?
A heatmap can do part of the job. It shows where the model looked, which is the right first form of explanation for an image-based severity model. A saliency map highlights the pixels a prediction used, but it does not explain how those pixels were used. Rudin makes the same point: knowing where the network is looking in an image does not tell you what it is doing with that part of the image.
TechnoLynx takes on this work through Generative & Agentic AI R&D, pairing each severity overlay with the grading standard it is checked against, and the first step is agreeing what the overlay must show for each decision tier.
So the overlay is evidence about localisation, not a full account of the grade. It becomes a justification only when it is attached to something the recipient can check it against, such as the grading standard the severity labels were built on. In car damage work that standard is the annotation guideline: it decides where a scratch becomes a gouge and where moderate becomes severe, and it is only defensible if practising adjusters agree with it.
The adjuster’s overlay and the policyholder’s explanation are different artefacts
Different recipients need different explanations of the same decision. EIOPA’s 2025 opinion on AI governance says insurers should be able to give authorities and auditors a global, comprehensive explanation of how an AI system works, while customers should, on request, get the system’s influence on a decision that materially affects them explained in simple, clear, non-technical language. Expert explanations are typically judged on technical accuracy and completeness, popularized ones for lay recipients on comprehensibility, persuasiveness and usefulness.
Our view is that the policyholder version should be written from the validated overlay and the grading rationale, not made by cropping the adjuster’s heatmap, and it should say plainly what the overlay can and cannot show. Gilpin and colleagues argue that offering a simplified description of a complex system to build trust, when users cannot understand its limitations, is inherently misleading.
What standard does a disputed automated settlement have to meet?
Neither the GDPR’s wording nor the NAIC bulletin sets a fixed bar for “explained enough”. The insurer has to set one, write it down and be ready to defend it. A review of AI explainability policy found no consensus on what makes an algorithmic explanation valid. The same review notes that the GDPR’s wording sets no baseline for the type or sufficiency of explanations. The NAIC bulletin says outright that its goal is not to prescribe specific practices or documentation, but to set out what information regulators expect insurers to produce when asked.
The anchors that do exist differ by jurisdiction:
- GDPR. Articles 13 and 14 require meaningful information about the logic, significance and consequences of automated decisions. Article 22 grants a right not to be subject to fully automated decisions. Article 15(1)(h) repeats the “meaningful information about the logic involved” requirement for access requests.
- The CJEU. In Case C-203/22, judged on 27 February 2025, the Court of Justice held that this right to meaningful information is a right to an explanation. It named the counterfactual method as one possible way to explain complex outputs, and it required a balancing against trade secrets.
- The EU AI Act. Enacted in June 2024, it demands transparency and documentation from high-risk systems. EIOPA reads its insurance high-risk category as risk assessment and pricing for natural persons in life and health insurance. For AI uses outside the high-risk category, EIOPA expects proportionate governance and risk management.
- US states. According to Altamira, the NAIC Model Bulletin was adopted in December 2023 and taken up by more than 25 US jurisdictions by early 2026, and New York DFS Circular Letter 2024-7 adds quantitative testing against protected classes. The bulletin itself lists what an insurer may be asked to produce during an examination: the techniques, measurements and thresholds used, plus validation, testing and model drift documentation.
Deciding what is “enough” means balancing what the regulator expects, what adjusters can use and what policyholders can understand. That balance is a judgement, and it belongs in the documented standard rather than being left implicit.
Our proposed explainability depth by decision tier
| Decision the severity score drives | Overlay requirement | Record requirement |
|---|---|---|
| Advisory: an adjuster grades the claim and the score is one input | Overlay sanity-checked before use | Basic usage records and quality monitoring |
| Automated settlement on eligible claim types | Overlay validated for faithfulness, and retested after material model changes | A richer record per claim, with renewed testing after material model changes |
| Declined, reduced or disputed payout | Validated overlay plus a plain-language explanation of how the AI contributed | Full reconstruction record, override reasons and, where appropriate, a second approval |
One point keeps the table honest. For image systems that cannot be comprehensively explained and have no suitable alternative, EIOPA expects complementary measures such as data governance and human oversight to make up for the lack of explainability.
The overlay is one exhibit in a reconstruction record
A rationale can describe why a system produced an output. It does not prove which data was used, which model version was active, which controls passed, who intervened or what the insurer finally did. For a challenged decision, the record should establish the use case’s risk level, the model identity and version, the authoritative inputs, the steps and rules invoked, the validation results, the human approvals or overrides, the action taken, and any later complaint linked to the case. How to set the tier in the first place is covered in validating models before they go live, from risk tiering to evidence packages. Assembling the case-level file (photo, overlay, score and threshold rationale) in a form an auditor can review is the subject of building a defensible evidence pack for audit and procurement review.
Explainability or interpretability: which does a claims model need?
For a convolutional severity classifier, the overlay is explainability, not interpretability, and the difference changes what you can claim in a dispute. Following Rudin, one 2023 analysis defines interpretability as how far a person can understand a predictor’s decision process by looking at the predictor itself. Explainability, by contrast, refers to methods that present otherwise hidden information about that process. A deep image model is not interpretable in that first sense, so any heatmap on it is a post-hoc explanation laid over a black box.
That has a cost. Rudin argues that post-hoc explanations cannot have perfect fidelity to the original model: if they did, they would be the model. NIST’s four principles of explainable AI make a related distinction. Explanation accuracy is separate from decision accuracy, so a correct payout can come with an explanation that does not reflect how the system reached it.
The two ideas are complements rather than substitutes, and each covers weaknesses in the other. In claims work, the grading guideline the network was trained to reproduce can serve as the interpretable layer, rather than the network itself. A documented guideline gives the overlay something external to be checked against. This is also where explanation turns into justification. Explanations describe how a decision was made, while justifications give reasons meant to make it acceptable. Justifications are normative: they rest on an external reference such as a legal or ethical norm. A claimant disputing a payout is asking for a justification, and the overlay only supplies part of it.
Can we generate a damage heatmap on every claim photo?
Yes. For a CNN-based severity classifier, Grad-CAM (gradient-weighted class activation mapping) produces such a map. The work is in proving each map reflects the model’s decision and agrees with the payout it accompanies.
TechnoLynx builds and validates that overlay layer for claims severity models as part of its AI Governance and Trust work. The first step is to define operationally what “the overlay drove the classification” must mean and which evidentiary bar it has to clear. After that, the engagement selects an attribution method that suits the model’s architecture, builds overlay generation into the inference pipeline at an acceptable latency for claims throughput, and runs alignment checks against expert-annotated damage regions.
Grad-CAM takes the gradients of a target class flowing into the final convolutional layer and produces a coarse localisation map of the image regions that mattered for that prediction. Its authors report that the maps are more faithful to the underlying model than earlier methods. A Grad-CAM map, like other saliency maps, highlights regions correlated with the prediction, and correlation is not causation. That distinction has to appear in any document that might reach a lawyer.
Architecture matters. Grad-CAM depends on a final convolutional layer, so a vision transformer severity model needs an attribution method adapted to its structure. One option is a perturbation-based method such as occlusion or RISE, which is model-agnostic because it only observes how outputs change when inputs are altered. Saliency methods also perform differently across network architectures and depths.
Faithfulness checks before an overlay enters a claim file
Run these checks on your own claims images and severity model. Method rankings from public benchmarks do not transfer reliably across datasets or hyperparameters.
- Model dependence check. Confirm the overlay depends on the model and the data; a method that fails such tests is inadequate for explaining what the model learned.
- Occlusion ground truth. Occlude parts of the photo systematically, measure which regions move the severity output most, and compare the overlay against that sensitivity.
- Deletion and insertion curves. Corrupt the top-ranked and bottom-ranked regions step by step. On a reliable map, the score should change much more when the top-ranked regions are removed.
- Localisation against expert masks. Measure how much of the salient area falls inside adjuster-annotated damage regions, for example with the IoSR ratio of intersection to salient region.
- Stability. Confirm that near-identical photos with similar outputs produce similar maps.
- Class sensitivity. Compare the maps for the highest-confidence and lowest-confidence grades. If they are nearly identical, the method is not separating one grade from another.
Two cautions apply to this list. Perturbation tests can push images out of the training distribution and distort the results. There is also no single agreed fidelity metric for vision models. This is why the documented standard should name the specific combination of tests used and their pass thresholds, rather than citing one score.
Will the overlay hold up if a regulator or claimant’s lawyer challenges the payout?
It holds up only as well as the documented validation behind it. Under challenge, the insurer should be able to show that this overlay method was tested against this model’s decisions and that its limits were disclosed. NIST notes that explanations can be built to match a classifier’s outputs while hiding its real behaviour. Altamira’s verdict on claims is blunt: a vehicle damage system that cannot show why it downgraded a claim is a compliance liability.
Keeping the overlay and the payout threshold consistent
Severity is the hardest output for automated damage assessment. In a ScienceDirect study cited by Altamira, detection reached 95.22% accuracy, location 76.48% and severity 57.89%. A production model therefore handles the claims it is confident about and routes the rest to a human, and the confidence threshold is a business dial, not a model constant.
What follows is our approach rather than an established method. Use the same expert-annotated validation set to set the severity thresholds and to measure overlay alignment, so that both rest on the same grading standard. Compute the overlay for the grade that set the payout, not for the top class in general. Send claims whose score sits near a threshold, or whose overlay falls outside the annotated damage, to an adjuster rather than settling them automatically. Setting the threshold value itself is a separate calibration exercise; what matters here is that an approver never sees a map that contradicts the grade.
Material changes to models or data sources should trigger renewed testing before they reach production. The NAIC bulletin expects evaluation of model drift to be documented as part of validation. Under this approach, every retrain reruns both the overlay checks and the threshold alignment.
Who can build and validate explainable damage overlays for our claims model?
TechnoLynx builds overlay generation into claims severity pipelines and validates it for faithfulness, localisation and consistency with payout thresholds, so that the explanation in each claim file can be defended when a decision is challenged. The engagement runs under AI Governance and Trust and starts by defining what the overlay must prove for your highest decision tier. Whether to keep this in-house depends on your team. If you already have computer vision engineers and an independent model validation function, the build is within reach. A specialist earns their place when the overlay must stand as evidence and nobody internally has run faithfulness tests before. When choosing a partner, ask to see the sanity checks, occlusion comparisons and localisation scores behind their overlays, not demo heatmaps.
Scope is limited in two directions. TechnoLynx supplies the engineering and validation evidence, not a legal opinion on a given jurisdiction. If a third-party vendor’s intellectual property blocks measures such as explainability, EIOPA expects you to mitigate with contractual clauses, external audits or due diligence testing. In that case, the vendor and an independent auditor may be the more realistic route. For a purely advisory model, a lighter internal review may be all the tier requires.
FAQ
Will AI take over claims adjusters?
Not for the claims the model is uncertain about. Production damage models settle the claims they are confident about and route uncertain ones to a human adjuster. Severity is the hardest output, at 57.89% accuracy in a ScienceDirect study reported by Altamira.
What counts as automated decision-making under the GDPR?
Article 22 covers decisions based solely on automated processing, including profiling, that produce legal or similarly significant effects on a person. A settlement an adjuster genuinely reviews before it is issued is treated differently from one issued with no human involvement.
