AI controller safety validation for a boiler setpoint controller comes down to six things that must exist before anyone removes human supervision. The first is a written safe operating envelope that matches the limits the live DCS enforces. The others are interlocks that do not depend on the model, a digital twin whose mismatch with the plant has been measured and bounded, a test matrix the plant’s engineers accept, shadow-mode evidence across the feedstock blends the plant will burn, and an evidence package that fits the plant’s management of change (MOC) process. None of this is a certification route. It is the engineering work that lets the people who own the plant’s safety case sign with full information.
The claim being built is a bounded one. A structured review of AI safety cases (arXiv, 2026) describes envelope-constrained claims: the system is acceptably safe only while it operates inside a validated envelope, and only if out-of-envelope behaviour is detected and the system moves to a safe state, for example by halting, degrading or handing off to a human. Every gate below supports one half of that sentence. Either it defines and validates the envelope, or it guarantees the detection and the fall-back.
Six gates for AI controller safety validation, in build order
| Gate | Evidence that closes it | Who owns the decision |
|---|---|---|
| 1. Envelope matches the live DCS | Line-by-line reconciliation of live trip, interlock and ramp-rate settings against the twin and the design-intent documents, with every deviation resolved | Control engineering confirms what is running; process safety confirms the limits |
| 2. Envelope specified and hazard-reviewed | Variables, numeric limits, margins and their derivation, plus a HAZOP extended to the AI loop | Process safety, control engineering and operations, not the ML team |
| 3. Independent enforcement | Interlocks on separate logic, I/O and field devices from the controller the AI writes to | Functional safety engineer and process safety team |
| 4. Twin mismatch bounded | Twin-versus-historian error at matched ramp events, compared with the margin to the trip boundary | Engineering, against an accuracy target agreed in advance |
| 5. Test matrix accepted | Code-based floor plus plant-flagged ramp, ambient and blend combinations | Plant engineering team as reviewer |
| 6. Shadow evidence sufficient | Per-blend, per-condition sample counts meeting a pre-declared gate, with no degradation trend | Plant operator, on written criteria |
The order matters. A test matrix run against the wrong boundary proves nothing, and shadow data cannot be judged until the envelope it is meant to cover has been written down.
Does the twin’s trip boundary match the one the live DCS enforces?
The envelope worth testing against is the one programmed into the running control and protection systems, and the plant’s control engineering and process safety functions sign its equivalence jointly: control engineering confirms the extract is what is running, process safety confirms those limits are the right ones. Design-intent documents describe what the plant was meant to enforce; the live configuration records what it enforces. A twin tested against design intent can pass every scenario and still approach a trip the real plant would take earlier, or one the real plant no longer has.
MOC guidance already lists the documents that record trip setpoints and logic. The Egyptian General Petroleum Corporation MOC guideline (EGPC-PSM-GL-018) gives the alarm and trip setpoints list, narrative safeguarding by instrumentation, and instrument loop and logic diagrams as example engineering deliverables for executing a change. Its review form asks whether a change touches as-built drawings, P&IDs, logic diagrams or cause-and-effect matrices. The IChemE Safety Centre puts the dependency bluntly: if you do not know a system’s existing hazards, its acceptable operating envelope and limits, or have no accurate P&IDs and equipment specifications, you cannot properly assess whether a proposed change is acceptable.
Run the reconciliation in three columns. The first holds the live configuration exported from whichever DCS and burner management system is installed, the second holds the twin’s modelled limits, and the third holds the design-intent documents. Every trip setpoint, interlock condition, time delay and ramp-rate limit gets compared across all three. Where the columns disagree, the deviation is logged and an owner resolves it before testing starts, because each disagreement is one of two things: an undocumented field change or a twin error.
Treat the as-built configuration as something that drifts. Summers’ SIS-TECH guidance on crediting BPCS protection layers lists management of change and revision control covering setpoints, fail configuration and operator overrides. The plant’s revision history is where drift shows up, and a configuration exported last year is not evidence of what runs today.
The AI team never signs this alone. Under IEC 61511 practice, deciding which functions are safety-instrumented and what integrity they need belongs to the functional safety engineer and the process safety team. The EGPC guideline describes change authorisers drawn from different departments (operations, engineering, maintenance) to give a multidisciplinary review and help keep it independent.
Which variables and limits define “safe”, and who signs them off?
Here, the safe operating envelope is a written specification. It lists the process variables the controller may never push past (drum pressure, steam temperature, drum level, furnace pressure, combustion stoichiometry such as excess oxygen), with numeric upper and lower limits and margins derived from equipment and process safety requirements. Process safety, control engineering and plant operations approve it, and the ML team is one contributor among several. DeepThermal, an offline reinforcement learning system for combustion optimisation in thermal power units (arXiv, 2021), modelled load, internal pressure and temperature satisfaction as safety constraints, and its authors note the need to keep internal pressure negative.
TechnoLynx works with the plant’s engineering and process safety teams to catalogue the controller’s variables, failure modes and existing protection. From that catalogue, TechnoLynx’s AI engineering for energy and utilities work turns the agreed limits into a testable envelope specification for formal sign-off, builds guardrail logic outside the model’s inference path, and runs the simulation scenarios that check both before deployment. The first step is that joint catalogue. Sign-off on the limits stays with the plant.
Can the limits be enforced independently of the model?
They can, and IEC 61511 requires it. IEC 61511 requires the safety instrumented system (SIS) to be independent of the basic process control system (BPCS), so that it can still act when the BPCS has failed. In practice that means separate logic solvers, separate I/O cards, and separate field devices with their own tags and wiring for any measurement used for both control and shutdown. For boilers, the NFPA 85 2022-cycle draft requires the burner management system to have independent logic, logic-solving hardware, I/O and power supplies, physically separate from other logic systems.
An AI controller that writes setpoints into the BPCS belongs to the control layer and should not be credited as protection against its own failure. Practitioner guidance on LOPA makes the general point: a BPCS loop is not an independent protection layer against scenarios involving its own failure. Summers adds a ceiling. Typical BPCS design and management practice limits risk reduction to one order of magnitude per controller, and claiming more from a single controller means designing and managing it as an SIS under IEC 61511.
The functional safety standards also give the model little room to carry the safety claim. A 2021 review of AI and technical standards notes that IEC 61508 does not consider the specifics of AI or learning systems. A 2022 analysis of the proposed EU AI Act reads IEC 61508 as not recommending AI at SIL 2 or above, because AI complicates properties such as freedom from intrinsic design faults and predictability of behaviour. The model can optimise. It cannot be the barrier.
Extending the HAZOP to the AI loop
HAZOP identifies the hazard scenarios and LOPA decides whether the existing layers bring residual risk to a tolerable level, so a HAZOP alone produces no SIL targets. Material changes to instrumentation or operating mode trigger HAZOP review through MOC. For an AI controller, add initiating events the original study never considered. Examples include the model driving setpoints to the edge of the envelope and holding them there, acting on stale or frozen inputs, oscillating against the existing loops, and behaving differently on a blend it was never trained on. The question for each event is whether a credited protection layer still covers it.
How far can the twin be trusted near the trip boundary?
Test feedstock blends the plant has not yet run through simulation, historian extrapolation or bounded physical trials, not unconstrained live operation. A twin is useful here because it can represent faults and undesired situations that are too risky and costly to stage on the real system. That usefulness has limits. Simulation of an unseen blend is an approximation, so it should be followed by cautious, incremental live validation.
Measure model-plant mismatch as the divergence between the twin’s predicted response and the historian record at matched load-ramp events, and judge it against the margin to the trip threshold rather than assume it acceptable. Validation practice judges adequacy by whether the measured accuracy is adequate for the intended application, and work on plant-model mismatch monitoring argues for index-based metrics that quantify its impact on control performance. Fidelity varies with ambient conditions (one twin study found environmental temperature significantly influenced twin performance). Where the historian lacks fast ramps or off-nominal ambient data near the boundary, the right call is to get that data before trusting trip-adjacent predictions, not to extrapolate.
Which load ramps, ambient conditions and fuel blends should the test matrix cover?
Use recognised codes as a floor and add the ramp, ambient and blend combinations the plant’s own engineers flag as likely to cause oscillation or trips. The matrix’s size is set by the reviewer’s written acceptance criteria, not by a scenario count. None of the codes cited here was written to define coverage for an AI setpoint controller, so part of the matrix is always interpretation.
The codes still give useful structure. ASME PTC 29 specifies tests for speed governors covering droop, deadband and deadtime, stability index, step response, gain and setpoint adjustment range and ramp rate, but its scope is hydraulic turbine governors. ASME’s Performance Test Code list includes PTC 70 on ramp rates, PTC 4 on fired steam generators, PTC 22 on gas turbines and PTC 19.1 on test uncertainty. Borrow PTC 29’s test types (step response and ramp-rate behaviour in particular) as the minimum shape of each scenario and use PTC 70 and PTC 4 to frame the boiler-side quantities, while stating plainly that none of them was written for this purpose.
Ramp rates need realistic levels. An IEA Clean Coal Centre report (Henderson) cites design ramp rates of 3 to 5 %/min for subcritical and 7 to 8 %/min for supercritical coal units, while EPRI-reported averages ran between 0.9 and 2.0 %/min, with maxima from 2.0 to 4.3 %/min. The same report notes that boiler and steam-side response changes significantly at low load, and that conventional control may not account for changes as a plant ages. The matrix should therefore reach the low-load corner, not just the nominal band.
| Matrix axis | Levels to include | Why it matters |
|---|---|---|
| Load ramp rate | Typical operating ramps, the unit’s design maximum, and fast ramps near the trip margin | These extremes are the ones that test the trip margin |
| Load band | Minimum stable load through full load | Boiler and steam-side response changes significantly at low load |
| Ambient condition | Seasonal extremes the site sees | Ambient temperature shifts the starting point a twin must match |
| Fuel blend | Every blend expected in the next operating season, including ones not yet burned | The envelope must hold across blends, not one |
| Disturbances | Sensor faults, stale inputs, mill or burner trips mid-ramp | These are the AI-specific initiating events from the extended HAZOP |
“Enough” is something to agree with the reviewer. The AI safety-case review found that acceptability criteria are often underspecified, with thresholds and failure budgets left undefined. The fix is to agree the pass criteria for each cell with the reviewer before any scenario runs, then iterate the matrix until the reviewer agrees it supports a field-trial recommendation.
How long must shadow mode run before supervision comes off?
Run shadow mode until the data covers the operating envelope, including every feedstock blend expected this season, with enough samples per condition to bound the control error. Elapsed time is the wrong measure. Two weeks of a single blend covers one cell of the matrix and says nothing about the blends coming in winter, so the shadow trial has to be extended across those blends before autonomy.
TechnoLynx defines the blends and conditions the shadow trial must cover and specifies per-blend statistical criteria (sample size, confidence bounds, degradation-trend checks). TechnoLynx then instruments the trial to collect that data, reports against the criteria whether removing supervision can be recommended, and supports a phased move to autonomous control once they are met. Work starts from the list of blends and conditions the trial must cover.
Shadow mode is safe for the same reason it is limited: its outputs never act. A practical guide to shadow deployment (EmiTechLogic) puts the whole value of shadow mode in the fact that it cannot make things worse, because the candidate’s output never reaches anyone. For a boiler, that means the shadow controller’s setpoints are logged next to the operator’s or the incumbent controller’s and never written to the DCS.
The same guide sets its own gate of 200 samples per arm that runs before any p-value is examined, because a significant result on a small sample is not the same claim as one on a large sample. It uses McNemar’s test on the discordant pairs, since paired predictions on the same inputs are not independent samples, and it warns that significance shows the disagreement is real without showing which side is right. Those examples come from ML serving, not plant control. Transferred to a boiler, each “arm” becomes a blend-and-condition cell. Each cell needs its own count and its own check for a degradation trend across repeated exposures to the same blend.
Some effects take time to show. A guide to shadow-testing credit models (StableRail) separates operational validation, which accumulates as soon as live cases flow, from outcome validation, which has to wait for each cohort’s performance window to complete. Expect the same split on a boiler: input feeds and setpoint disagreement visible from day one, slower blend-related effects needing time on each blend.
Shadow mode cannot prove closed-loop behaviour. StableRail notes that because the shadow decision is never applied, the test cannot measure the effects that acting on it would cause. Passing shadow should lead to a limited rollout with monitoring and a route back to the incumbent process, not an immediate full switch. Rare blend and condition combinations may never appear even in an extended trial, and that residual risk belongs in the safety case.
What happens when the model asks for a setpoint outside the envelope?
Have a layer reject or clamp the setpoint automatically, before it reaches an actuator, checking it against fixed limits and never consulting the model’s confidence. No one needs to intervene in the moment, and the model is never the last line of defence.
Google DeepMind’s AI control of data-centre cooling is one published example. Actions computed by the AI are checked against operator-defined constraints, and the local control system then verifies the instructions against its own set of constraints, so operators keep full control of the operating boundaries. Operators can exit AI control at any time, and control transfers to the on-site rules and heuristics. DeepMind also deliberately narrowed the optimisation boundaries to a smaller operating regime, accepting less energy saving in return. Its system also discards low-confidence actions. That is a useful model-side filter, but it is not the independent layer, because it fails whenever the model is confidently wrong.
For boilers, NFPA 85 supplies the hard backstop. Any logic command from the burner management system takes precedence over the combustion control system. No single component failure in the logic system may prevent a mandatory master fuel trip, and the operator has a dedicated manual switch that trips independently and directly. Signals that initiate mandatory master fuel trips must be hardwired.
Clamping is not automatically safe. Work on MPC-based safety filters for reinforcement learning (arXiv) defines an unsafe state as one past a “point of no return”, where a future constraint violation is physically inevitable whatever control follows. A check that looks only at the instantaneous action may intervene too late in high-inertia systems, which is why that work pre-computes the feasible boundary offline and projects each action into it before execution. A boiler has a lot of thermal inertia. The rejection limits for AI setpoints should therefore sit inside the trip settings by a margin derived from plant dynamics, and whether to clamp or hold the last good setpoint has to be checked case by case. DeepThermal’s cost critic models constraints as penalty costs, which looks like the kind of soft constraint the MPC safety-filter work says optimises expected satisfaction without a step-by-step guarantee.
The override itself needs validating before it goes into service. Summers calls for any change affecting safety interlocks to be examined for validation before the function goes into service, and for access to engineering interfaces to be controlled. The override must not be something the software stack running the model can disable. Bypass and reset conditions belong in the specification as well.
The overnight question needs a written answer: named on-call roles, response times, the handover from AI mode back to the incumbent controller, and a drill schedule so the runbook stays current as staff change. One limit applies to everything in this section. An envelope check catches only setpoints outside its limits, so behaviour that stays inside the limits but is otherwise wrong needs the monitoring described below.
How should the validation evidence be packaged for MOC and safety-case sign-off?
Start from the plant’s own MOC template and safety-case requirements. Structure the report so that each section answers a step the review board already runs, and include the guardrail and interlock design in the package instead of leaving it to the plant. Board approval rests with the board, however good the documentation is, but a package that matches their process removes the reasons to send it back.
Installing the controller is clearly an MOC event. The IChemE Safety Centre lists changes outside the safe operating envelope, changes to instrument setpoints and control system changes (logic, interlocks, PLC or DCS programming) as changes that need MOC. Its screening questions ask whether an interlock or safety instrumented function is added, modified or bypassed, and whether process parameters could stray outside the defined envelope. The same document places changes to process control and tuning parameters within a defined safe operating envelope out of MOC scope. That boundary is why the envelope definition matters so much. Once the board approves it, setpoint movement inside it can become routine operation, while the change that installs the controller and its interlocks goes through full MOC.
IChemE’s temporary-change category covers trials that affect operating conditions, held to the same rigour as permanent changes, with escalation if the time limit is extended. The EGPC guideline classes changing process control or alarm functions as a Level 3 significance of change, which calls for more rigorous studies such as HAZOP. Its technical basis for change should address what is changing and how, what it will achieve, how, and whether it is safe and why.
| MOC step | What the validation package supplies |
|---|---|
| Initiate and screen | Scope of the change, the controller’s variables, and answers to the screening questions |
| Technical basis | The envelope specification, the live-DCS reconciliation and its sign-offs |
| Risk assessment | Extended HAZOP findings, and LOPA impact for any credited protection layer the change touches |
| Approval | Test matrix results, the twin mismatch analysis, shadow evidence against pre-declared criteria, and the interlock and override design |
| Pre-startup safety review | Verified installation, updated procedures and completed training before start-up |
| Close-out | Updated documentation and communication to affected stakeholders |
Approval-grade evidence is an engineering output, built so that a non-engineer can read it, defend it and sign against it. It does not replace the plant’s sign-off or any certification. The safety-case review lists the quality criteria such evidence should meet: independence, coverage, recency, reproducibility and representativeness. How to assemble artefacts like these into a reviewable pack is covered in more depth in our guide to building a defensible evidence pack for audit and procurement review.
Including the guardrail design needs a written responsibility split. State in the package which party designed, verified and maintains each protection layer.
The part no interlock covers
The interlocks bound the worst case. The 2021 standards review describes ISO/IEC TR 5469 on functional safety and AI systems as still in development, with its idea of probabilistic risk acceptance “as promising as it is controversial”. Until that settles, we would treat retraining as the obligation that matters most after go-live. The safety-case review describes continuous-evolution claims that hold only if each update passes a formal change-control procedure, shows no significant degradation against the prior version and leaves an audit trail of revalidation. NASEM’s 2023 digital-twin report makes the same argument for the twin: verification, validation and uncertainty quantification must be a continual process that adapts as the plant, the models and the data change. In practice, a retrained model or a new feedstock is a new entry in the MOC log, not a quiet redeployment.
FAQ
What do upper and lower control limits mean?
In statistical process control, upper and lower control limits are bounds calculated from a process’s own variation, typically three standard deviations either side of the mean, and a point outside them signals that something unusual has happened. They are different from the safety limits in an operating envelope, which come from equipment and process safety requirements and are enforced by trips and interlocks.
What is the difference between design verification and design validation?
Verification checks that a design was built to its specification, while validation checks that the specification itself meets the real need in the intended use. For an AI setpoint controller, the envelope reconciliation and interlock tests are mostly verification, and the shadow trial and field trial are validation.
