Secure firmware updates need more than a valid signature, because a signature proves who built an image, not that the device owner authorised it to run. Trojanised updaters signed with legitimate certificates such as “ASUSTeK Computer Inc.” stayed undetected for so long partly because of that signing. Signing has to be backed by three more controls. Remote attestation compares measurements of what devices run against reference values. Staged rollouts can be set to halt and roll back automatically. Hardware-backed privilege separation keeps one bad component from reaching the whole device.
The first step is to frame the threat model and the constraints of the existing hardware.
Why doesn’t a valid signature stop a malicious firmware update?
A signature check answers one narrow question: did someone holding this private key sign these bytes? NIST SP 800-193 separates the two ideas explicitly. An update is authentic when its source and integrity can be verified. Authorisation is the permission to perform the update, and it is typically rooted in the device owner rather than the manufacturer. The same guidelines define an authorised update mechanism as one that checks for approval before installation, for example possession of a credential or confirmation by a physically present person.
A validly signed but unauthorised image reaches a device by routes including these three:
- A stolen or misused signing key. NIST notes that an attacker holding the private key could sign invalid firmware that damages devices or injects malware.
- Signing access abused without stealing the key. The TUF authors describe attackers signing several malicious OpenSSH versions with the Red Hat signing key, which Red Hat believes was not itself compromised. Afterwards there was no secure way to revoke the key or clients’ trust in the packages.
- An old, genuinely signed image with known vulnerabilities, replayed onto the device. NIST requires update mechanisms to be able to block unauthorised updates to an earlier authentic version that has a security weakness.
RFC 9334 warns that a manufacturer’s endorsement may authenticate a device, but it is not enough on its own to authorise that device for a specific purpose. The OCP secure boot specification adds that secure boot should not rely on update authentication done only in firmware, because firmware vulnerabilities can be exploited and then persisted.
Signing still does its job: it narrows who can produce an image that verifies. Signature verification checks authenticity, not whether the release process approved the image. Once a rogue image verifies, a signature check will never flag it. Google adds a second, independent record for Pixel images. If an attacker signs a modified Pixel image with Google’s key, anyone checking Google’s binary transparency log will find the image missing and know not to trust it, because publishing to the log is a separate process from signing.
How do we catch a device running a signed image it should not be running?
Remote attestation detects it remotely. The device reports measurements (hashes) of the code it loaded, and a verifier compares them against reference values (known-good measurements). In the IETF RATS architecture (RFC 9334), the device is the Attester and produces Evidence. A Verifier appraises that Evidence against policy, and a Relying Party, such as the update or access service, decides what the device may do.
For IoT and embedded devices, TechnoLynx builds an attestation scheme that fits the device’s battery and connectivity and checks how fast it catches a rogue image across the fleet.
Measured boot builds the evidence in layers. ROM measures the bootloader, then the bootloader measures the kernel. The claims about each layer are signed or stored by the layer below, so the bootloader cannot rewrite the claims about itself. MCUboot’s measured boot option adds a boot record to each signed image that carries the software type, version, signer ID and image hash. The OCP recovery specification lists every firmware image on the device, including stored recovery images, as included in the reported attestation results.
The design decision that matters most is where the reference values come from. If the verifier accepts any image that carries a valid signature, attestation inherits the signature’s blind spot. The reference set has to come from the release approval record: the specific builds cleared for this fleet and this ring. Uptane uses this pattern for vehicles. Each ECU signs a manifest of what it installed, and the server-side director flags any mismatch between what it sent and what was actually installed.
Attestation has one limit to accept from the start. The OCP specification notes that appraisal of measured boot normally happens after the code has run. Attestation normally detects a rogue image after it has run, rather than stopping it from running. Its value depends on what the fleet does with a failed result. RFC 9334 lists the options: devices that cannot be attested to a good state can get reduced privileges, be taken out of service or be flagged for repair.
Which attestation scheme fits a battery-backed smart lock?
Choose the scheme by power, connectivity and the threat you need to cover. One literature review sorts attestation into software-based, hardware-based and hybrid schemes. Hardware gives strong guarantees at a cost, and hybrid designs use a minimal hardware anchor to stay cheap and still secure.
| Device situation | Scheme | What it gives you | What it costs |
|---|---|---|---|
| Battery-backed, intermittent connectivity (smart lock) | Periodic self-measurement, collected when the device connects (ERASMUS style), on a hybrid anchor | A history of measurements since the last collection, with almost no prover computation at collection time | Evidence reaches the verifier only when the device connects |
| Minimal MCU with no TEE or secure element (RISC-V) | Hybrid anchor built from ROM and Physical Memory Protection (LIRA-V) | Attestation without a TPM, secure element or separate privilege modes | Trusts the PMP primitive and ROM for key secrecy and measurement integrity |
| Device reachable for on-demand checks | Measured-boot evidence with verifier-issued nonces, or on-demand attestation around updates | Fresh evidence immediately before and after an update | An extra attestation at retrieval time, with timekeeping left to the verifier |
| Threat model includes tampering after boot | Runtime control-flow attestation on TrustZone-M (ISC-FLAT) | Detects control-flow hijacking and code reuse at runtime | Larger computational overhead and more complex protocols |
| No hardware anchor at all | Software checksum attestation | Cheap and easy to deploy | Relies on strong assumptions about the adversary and gives minimal guarantees |
ERASMUS was designed as an alternative to on-demand attestation for low-end devices. The ERASMUS authors report that one self-measurement, based on keyed BLAKE2S, takes under 300 ms on a 1 GHz device with 10 MB of memory. The measurements can sit in unprotected storage, because malware without the key cannot forge them, and any tampering shows up at the next collection. The US-AID authors measured energy per protocol run as low as 1 mJ for an attestation and 5 mJ for a join or heartbeat run on a MICAz device with up to four neighbours, constant in network size. Energy grows linearly with the number of heartbeat intervals, so the interval is a direct trade between detection accuracy and battery life. RFC 9334 allows flexibility on freshness so that evidence can be cached and reused, which it calls especially valuable where generating evidence uses a large share of a constrained device’s energy budget.
How often to attest should follow how valuable the device is to an attacker. A strategic attestation study argues that low-value devices need not be checked often while high-value ones may need frequent attestation. For a lock, we would tie self-measurement to the device’s existing wake cycles, collect on every connection, and add one on-demand attestation immediately before and after each update, a use the literature review names for the ERASMUS on-demand extension.
How quickly can attestation surface an anomaly across a 300,000-unit fleet?
Time to detection has two bounds: how often devices report, and how fast the backend can appraise what arrives. Detection lag cannot be shorter than the time between measurements and the time between collections.
The slimIoT scalability figures come from simulation. In the slimIoT simulation of 100,000 devices in eight clusters, checking every device for physical attacks plus verifying the software of two clusters (25% of devices) took about 30 seconds in a binary tree, under 17 seconds in a 4-ary tree and 9 seconds in an 8-ary tree. Full attestation took at most 18 seconds (8-ary) and 50 seconds (binary). Those numbers describe a mesh network in simulation. Treat them as evidence that aggregation scales, not as a latency forecast for a 300,000-unit fleet on cellular or Wi-Fi check-ins. The slimIoT authors also note that skipping clusters in a round saves traffic but raises the latency of finding compromised devices in those clusters. Aggregated swarm schemes have another catch: they verify the whole swarm but cannot tell you the state of individual devices.
On the backend, the hard trade-off is minimising false alerts while keeping attestation useful. A study of the Keylime continuous-attestation framework found that false positives were mainly caused by OS updates that no longer matched the predefined policy. The fix was to generate a new policy before each scheduled update. For firmware, that means publishing the new reference values as part of every release, before the first canary receives the image. The same study found that Keylime stopped polling after a discrepancy, which opened a window for an attacker. The authors recommend always completing the full attestation instead. They also argue that attestation should verify that a known list of executables is intact, not act as an intrusion detection system.
How do we design a staged firmware rollout so a compromised image reaches only a small share of the fleet?
A staged rollout reduces the blast radius of a compromised image. Release to a small canary cohort, hold, and expand only while telemetry agreed in advance stays inside its thresholds. One OTA rollback guide from Defenders.cloud recommends increments of 1%, 5%, 10%, 25%, 50% and 100% with explicit hold points. It gates each step on crash rate, boot success, update completion time and support ticket volume.
TechnoLynx sets canary size and halt thresholds against how many devices a bad image may reach, builds the halt, quarantine and rollback logic, and tests anti-rollback behaviour against compromised-image scenarios.
The canary has to be big enough and run long enough to be representative. Google’s SRE workbook warns that a handful of samples gives no useful signal, and that metric intervals must be no longer than the canary itself. The same workbook advises running only one canary deployment at a time. Promote the exact signed artifact you tested, because rebuilding between environments invalidates the evidence. CrowdStrike’s root cause analysis of Channel File 291 describes the pattern as a mitigation: canary testing, then successive deployment rings, each followed by bake-in time while telemetry is gathered. Ring sizing has a tooling caveat. Mender calculates phase sizes from the dynamic group at the moment the deployment is created, and a device is updated whenever it next polls, so devices that join late can land in the first phase.
Reliability metrics alone are not a security gate. A malicious image that boots and runs normally passes crash-rate and boot-success gates, so the security gate has to include attestation results and integrity telemetry, not only stability. Security telemetry already appears among the predefined go/no-go signals in one enterprise rollout guide. For the general mechanics of OTA rollback, see our guide to designing safe over-the-air updates and rollback across fleets.
What telemetry thresholds should automatically halt a firmware rollout?
Acceptable behaviour has to be defined per metric for each system. A threshold trades false halts against missed bad deployments. Google’s SRE workbook says it plainly: an overly strict definition of acceptable behaviour produces false positives, and a loose one lets a bad canary through. Alongside comparisons against a control population, the workbook recommends absolute measures such as defined SLOs. Set the thresholds and name the person who can pause or roll back before the first canary, so nobody redefines success after seeing inconvenient results.
AWS IoT Jobs and Azure Device Update implement halting as follows:
| Mechanism | Trigger | Effect on devices |
|---|---|---|
| AWS IoT Jobs abort configuration | A threshold percentage of devices report REJECTED or FAILURE | Cancels the job; devices must be programmed to report the failure status |
AWS IoT STOP_ROLLOUT (default) |
Abort criteria met | No new devices notified; queued and in-progress executions run to completion |
AWS IoT CANCEL |
Abort criteria met | Queued executions cancelled; in-progress ones continue |
AWS IoT FORCE_CANCEL |
Abort criteria met | Queued and in-progress executions both cancelled |
| Azure Device Update automatic rollback | Failure percentage and a minimum number of failed devices | All devices in the group rolled back to the selected update version |
Two judgements follow from this. First, pair any percentage with a minimum device count, because on a small first ring one flaky unit is a large percentage. Second, an attestation mismatch on a device that received the new image should be a zero-tolerance gate that halts the rollout. Reliability metrics can be compared against the control population. Halting a rollout does not undo installs that have already happened, so the rollback path has to be designed separately.
Can quarantined devices be forced back to a known-good image?
Yes, if three things hold. The device must still be reachable by a path the compromised image cannot block. The rollback image must be signed and allowed by the version policy. The anti-rollback counters must accept it. The Defenders.cloud guide puts the second condition as signed rollback images, explicit version policies, audit logs and approved exceptions for emergency downgrades, so that rollback stays a recovery mechanism rather than a downgrade vulnerability.
Quarantine comes first. It stops further updates, removes the device from rollout cohorts and can tighten identity and access controls until the device verifies again. The OCP recovery specification has the platform root of trust start recovery when attestation detects tampering, when a device stops responding, or on forced recovery. Forced recovery can be commanded administratively, and NIST SP 800-193 says devices should give administrators a way to force recovery.
A compromised image can simply ignore OTA commands, so the recovery path has to sit below it. OCP requires that a device which fails to load a valid image keeps trying over its permitted firmware load interfaces, and that every recovery path still verifies signatures.
Anti-rollback counters decide what “known-good” can mean. Chromium OS keeps a Key Version and a Firmware Version in TPM NVRAM that only the read-only firmware can write, and it refuses any image whose key index or firmware version is lower than the highest seen. MCUboot compares each image’s security counter against a value held in non-volatile, trusted storage. A previous build with a lower version is refused by these checks. In practice, forced rollback is a roll-forward: rebuild the known-good code with a new, higher version and sign it. That matches the forward-fix route one rollout guide recommends when a change to firmware or security controls makes a downgrade unsafe. NIST warns that a recovery policy which reinstalls the version that was attacked can lock the fleet into an attack-and-recovery cycle.
The bricking risk sits in when the counter moves. MCUboot boots a new image as a test swap and reverts on the next reset unless the image marks itself OK. AOSP marks a slot successful only after the new system passes its post-reboot checks. Advance the stored counter only after that confirmation. Otherwise a failed boot falls back to a slot the counter now rejects, and the device has nothing it is allowed to run. For recovery images, OCP recommends A/B versioning so the recovery copy is never older than N-1. A dedicated C-image is signed with the production keys and obeys the same anti-rollback rules.
How do we re-architect the image so a compromised build cannot reach every subsystem?
Split the image into privilege domains with hardware-enforced boundaries, so a signed-but-malicious component in one domain has no path to the camera sensor, storage and network at once. The TUF authors describe this as applying least privilege to installed updates, so that a malicious update stays contained and a later update can restore a secure state. NIST SP 800-193 asks devices claiming protection conformance to run critical platform firmware isolated from software, logically or physically, for example in TrustZone.
TechnoLynx frames the threat model and the isolation boundaries for the camera, storage and network subsystems. TechnoLynx then chooses the separation architecture the existing SoC can support, implements the partition retrofit and peripheral access restrictions, and validates containment with targeted compromise simulation before supporting the rollout of the new layout.
The threat is concrete. The SANCTUARY design for TrustZone explicitly assumes a user may be tricked into installing a malicious but valid application, and limits it to user privilege and its own memory so that even a privilege escalation cannot reach secure-world or other normal-world memory. NIST’s “Resilient” level says even non-critical devices should be designed so that their compromise does not affect the security of the platform as a whole.
What is the smallest set of privilege boundaries to add first?
Add three boundaries first: a secure world, separated partitions, and per-domain access to memory and peripherals.
- A secure world for keys, update verification, attestation and the anti-rollback counter. In TrustZone, secure-world software can reach the whole SoC, while normal-world software reaches only hardware marked non-secure. Roots of trust are typically protected by running them isolated at a higher privilege than anything that could modify them, and RFC 9334 assumes the attesting environment is isolated from the environment it measures.
- Separate partitions for boot, root filesystem and application. Mender describes splitting these into distinct partitions, each with A/B versions. A read-only system partition checked by dm-verity guarantees the device boots an uncorrupted image.
- Per-domain memory and peripheral access. Arm’s TZC-400 filters memory transactions by bus-master ID, and every bus master (CPU, GPU, DMA controller) carries one. On MCUs, Zephyr user-mode threads see only their own stack, program text, read-only data and the memory-domain partitions granted to them. TrustLite-style execution-aware MPU rules play the same role on smaller parts. OCP recommends devices disable bus mastering at power-on until the platform root of trust judges them healthy.
A sensor or ISP block that can DMA anywhere in memory goes around a carefully configured MPU, so bus-master filtering matters as much as CPU-side protection.
There is a hard limit. If the hardware has no secure execution environment, containment is limited to OS-level sandboxing, and full protection probably needs a hardware revision. RFC 9334 makes the strength of the root of trust the deciding factor: mutable software, read-only-after-boot firmware or immutable ROM. The Trusted Firmware-M documentation warns that if the first-stage bootloader and root public key are not immutable, secure boot can be bypassed and arbitrary code executed. A software-only scheme cannot contain a compromise if the component that holds the root of trust is reachable from the compromised one.
Can we retrofit privilege separation through an A/B partition scheme without a hardware respin?
Partly. A/B slots give you a fallback slot that a bad image does not overwrite. They do not isolate subsystems that are running at the same time. AOSP’s A/B scheme keeps the unused slot untouched by the running system and reboots into the old slot if a new one fails to boot. Uptane uses extra storage the same way, so a tampered image cannot overwrite the previous known-good one.
The cost is storage. Mender notes that A/B updates keep two full system images, effectively doubling storage, and that devices with 512 MB or more typically support it. Uptane describes a cheaper alternative that stores only a delta of changed blocks.
Changing partition layout over the air without new hardware has precedent. AOSP ships retrofit OTA packages that bring dynamic partitions to A/B devices that launched without them, mapping the new layout onto the existing physical partitions while keeping the GPT. TF-M with MCUboot can boot multiple images, with the secure and non-secure images signed and updated independently. That lets the secure image be updated on its own. The retrofit stops at what the SoC’s secure boot and TrustZone or secure element actually support. If those are missing, the answer is a respin.
Choosing the first control for a fleet already in the field
Each control covers a gap in the others. Attestation normally detects a rogue image only after it has run. Rollout gates limit the blast radius and judge each ring against predefined telemetry thresholds. Isolation limits what a bad component can reach but tells nobody it happened. Privilege separation is architecture work and should be scoped against what the SoC can enforce.
Two related questions are covered separately. The first is the signing side: hardware-anchored keys and key hierarchies that keep one compromised key from validating images fleet-wide. The second is how to prove to an auditor that secure boot and anti-rollback are enforced on production units, not just on a development board. For both, see our guide to anchoring OTA signing and attestation to a hardware root of trust.
FAQ
Can malware be found in firmware?
Yes. NIST SP 800-193 warns that attackers may inject persistent malware into platform firmware to disrupt operations or exfiltrate data. Attestation compares measurements against known-good reference values, and it cannot judge unmonitored code.
What happens if I don’t update my firmware?
The device stays exposed to the known vulnerabilities that security updates would fix. The TUF authors describe attackers deliberately blocking updates, which they call a freeze attack, precisely to gain time to exploit those weaknesses.
