Published studies report no standardised testing method for physical adversarial attacks, so a resistance claim you can defend has to be built rather than bought. Building it takes three things. The first is a fixed, logged test protocol for each sensor type, run on the production hardware. The second is a detection layer that tells an attack apart from ordinary degradation. The third is a pass threshold derived from measured error distributions and argued inside a recognised safety or biometric framework.

Why no single benchmark covers physical spoofing

Several research groups report the same gap. A study of physical adversarial patches on face recognition found no systematic testing protocol for physical attacks: evaluations typically relied on a handful of volunteers in one environment, which made methods hard to compare or reproduce. The PADetBench authors reviewed more than forty physical attack methods and found most were run under unaligned conditions without fair comparison. A systematisation of sensor spoofing research describes the literature as ad hoc, with researchers picking the state they want to corrupt and then designing a spoofer for it.

Each sensor is attacked through its own channel (RF signals for GNSS, timed laser pulses for LiDAR, acoustic resonance for MEMS inertial sensors, printed stickers or projected light for cameras).

What you test against instead is a combination: published attack baselines for your sensor, the standard metric where one exists (biometrics has ISO/IEC 30107-3), and a threshold tied to the consequence of failure. Enumerate the threat set first. For that step, see our guide to threat modelling the inference pipeline, from GPU isolation to sensor spoofing.

Attack classes and spoofer effort by sensor

The table summarises what the cited studies report about how each sensor is attacked and what the attacker needs.

Sensor	Attack classes	Equipment and effort reported
GNSS (GPS)	Simulator broadcast; portable receiver-spoofer that aligns counterfeit signals, then drags position or time	Stand-alone civil receivers can be spoofed with a signal simulator, a power amplifier and an antenna; a portable receiver-spoofer built from off-the-shelf parts is harder to detect
LiDAR	Laser injection synchronised to the victim’s pulses; shape manipulation; object placement	Photodiode, delay component and infrared laser; about 60 points reliably spoofed within an 8° horizontal window in one study, up to 200 points with better optics in another
IMU	Acoustic resonance injection; magnetometer field spoofing	A $320 function generator, amplifier and tweeter near 110 dB; attack distance up to 37.58 m with a long-range acoustic device; magnetometers spoofed with controlled coils
Ultrasonic and mmWave radar	Signal injection	A $20 Arduino for ultrasonic; a 76 to 77 GHz radar spoofer can cost more than $10,000
LoRaWAN telemetry	Replay through a wormhole, adaptive data rate spoofing, Class B beacon spoofing	Inexpensive off-the-shelf LoRa hardware
Fingerprint and face	Printed, moulded, replayed or masked presentation attack instruments	Level A: up to a day, little expertise; Level B: up to a week, proficient, standard or specialised kit; Level C: expert, bespoke, a month or more

Each protocol should record the variables the cited studies show results depend on. For GNSS, one detection metric pairs carrier-to-noise density (C/N0) with calibrated received power. Camera patch results depend on distance, patch size, placement and measured light level. LoRaWAN results depend on the specification version and the network server software. Biometric results need the threshold, SDK version and device.

Can someone fake the GPS or occupancy signal on our vehicles without us noticing?

For GPS, yes. Civil GPS signals carry no signal authentication, which is what makes them spoofable. How much equipment and skill an attack on your sensors and telemetry link would take is known only once a dedicated adversarial test on representative hardware shows it. A generic IoT security review will not answer the question.

TechnoLynx can run that test in four steps, as part of its AI engineering for IoT and embedded devices: framing what an inflated-ridership or false-position attack looks like, building the spoofing, replay and injection rig on representative onboard hardware, measuring the effort each successful attack needed, and classifying each finding as a device, pipeline or process gap.

The GPS threat is a continuum. The Cornell portable-spoofer study splits it into simplistic, intermediate and sophisticated attacks. It rates the signal-simulator attack the greatest near-term threat, although known defences against it can be implemented in receiver software, and rates the portable receiver-spoofer the greatest long-term threat. Inertial cross-checking does not close the gap on its own. In a study of GPS/INS tracking, attackers generated fake signals with GPS-SDR-SIM and picked spoofed routes with matching curvature and turn angles. On at least 10% of routes in every city tested, they reached points 30 km from the assigned destination without detection. The same team built a coil-based device that spoofed the magnetometer. Authenticated navigation messages close only part of the gap: Galileo’s OSNMA service definition says it authenticates the navigation message but does not by itself guarantee an authentic navigation solution, which depends on the receiver implementation.

Telemetry: what LoRaWAN’s MIC does and does not stop

If occupancy counts or positions travel over LoRaWAN, the protocol version changes what a replay test has to try. The ChirpOTLE researchers note that the message integrity code (MIC) stops a wormhole attacker from altering message content, but the attacker can still change metadata such as timing, location, SNR and RSSI. Under LoRaWAN 1.1, the channel and data rate are part of the MIC calculation, so a replayed uplink has to use the same channel and data rate; LoRaWAN 1.0.x allowed replay with different transmission parameters. Class B beacons cannot be meaningfully encrypted or authenticated by design. The same team showed that gradually shifted fake beacons could disrupt Class B downlink in an area in under ten minutes.

Can devices already in the field be hardened with payload signing or tighter MIC checks? Only partly. A systematic review of LoRaWAN security notes that the protocol has no over-the-air firmware update, which makes firmware changes expensive and leaves protocol versions coexisting in one network. Fleet-wide flagging of stuck or replayed values, and embedded AI anomaly models small enough to run on a LoRaWAN gateway, are separate pieces of work that follow from what the test finds.

Reading the result as a hardware, pipeline or process gap

Where an attack succeeds tells you which kind of fix you need. Success at the sensor or firmware level (no signed sensor data, no GNSS anti-spoofing) is a device trust and provisioning gap. Success by injecting or modifying messages in transit is a reporting-pipeline gap that needs authenticated telemetry. An exploit that needs physical access or operator collusion is a process gap that needs audit controls. A clean result covers only the tested attack surface and firmware version. Extra GNSS cross-checks against cellular or Wi-Fi positioning also cost money, which may not be justified when nobody has much incentive to cheat.

What test methodology shows our fingerprint and NFC sensors resist replayed or synthetic spoofs?

No test proves a sensor cannot be spoofed. What a documented protocol can do is report error rates against a defined, reproducible set of attack instruments, which is something a security reviewer can check. For the fingerprint half, the reference is ISO/IEC 30107-3, the international standard for testing and reporting biometric presentation attack detection (PAD).

TechnoLynx can scope the attack set against your customer’s threat model, assemble the presentation attack instruments and replay equipment, run and log the protocol, compile the documentation for the security review, and retest after any fix.

The ISO/IEC 30107-3 vocabulary

ISO/IEC 30107-3 measures detection with two error rates. APCER is the proportion of attack presentations using the same PAI species that are wrongly classified as genuine. BPCER is the proportion of genuine presentations wrongly classified as attacks. A PAI (presentation attack instrument) species is a class of instruments made with a common production method. For a full system, where the question is whether the attacker gets in, the metric is IAPAR: the proportion of impostor attack presentations of one PAI species that result in an accept. Because the security and convenience errors trade off as the threshold moves, the standard allows reporting BPCER at a fixed APCER, for example BPCER20 at an APCER of 5%.

Equipment and attack samples

The FIDO biometric certification requirements set out a concrete recipe. Level A fingerprint PAIs are printed images, with each printer make and model counted as a separate species. Level B PAIs are casts from moulds made by etching a circuit board, laser etching, printing on transparency, or using a 3D printer under $500 with XY resolution finer than 0.05 mm, cast in gelatin, silicone, play-doh and similar materials. Additives such as graphite create further species. The FIDO procedure has an accredited lab select six Level A and eight Level B species, enrol 15 subjects, and run 10 impostor attack transactions per PAI per enrolment. Each PAI is first checked for validity, the threshold is fixed at the vendor’s claimed false accept rate, and every interaction, including threshold changes and sensor cleaning, goes in a log.

The set of materials tested defines the coverage. The LivDet competitions added materials the algorithms had never seen because a real attacker’s material is unknown, and they observed performance drops whose size depended unpredictably on the material. Even colour mattered: one system accepted pink and lime-green play-doh spoofs as live for almost all images, which produced a 28% error rate for that material.

The NFC half, the deadline and who runs the test

ISO/IEC 30107-3 and the FIDO procedure address presentation attacks on the biometric sensor. The NFC relay or replay part of the protocol has to be designed from your threat model, with the rationale written down alongside the results. On timing, the FIDO design produces 150 attack transactions per species (15 subjects × 10). Its 7% IAPAR level allows at most 10 errors out of those 150. Fitting a review into a few weeks by cutting species or transactions weakens the statistical confidence of the result. FIDO certification requires the testing to be performed by an accredited independent laboratory. Confirm early whether your customer’s review accepts first-party evidence.

When is the biometric SDK vendor’s liveness check not enough?

The vendor’s check is not enough when the vendor cannot show which attacks it was tested against, at what threshold and by whom, or when your threat model includes attacks its liveness detection was never designed to catch. Ask for a written conformance statement. For comparison, one face liveness SDK vendor states that its passive liveness detection covers printed photos, screen replays and masks, and was tested to iBeta Level 2 against ISO/IEC 30107-3. That is the vendor’s claim, not your measurement. Another vendor, Innovatrics, documents that its passive liveness does not cover video injection or deepfakes. That kind of boundary statement is what you need from your own supplier. The FIDO requirements also ask vendors to document the protections around the sensor against injection and replay.

Vendor numbers do not carry over to your device automatically. Innovatrics’ documentation says liveness scoring is model-specific, with no single global threshold, and should be tuned on representative data. The other vendor advises testing with your actual cameras and lighting and logging the threshold, SDK version and device so results can be reproduced. Independent measurements exist as well: NIST’s FATE PAD report (NISTIR 8491, September 2023) quantified the accuracy of 82 passive, software-only face PAD algorithms.

Does a Cortex-M33 secure enclave already cover presentation attacks?

No. Arm describes TrustZone on the Cortex-M33 as a way to achieve software isolation and a security foundation for IoT devices. An enclave like that isolates software and gives the device a security foundation. Nothing in that description covers judging whether the finger or face at the sensor is live.

The decision follows from the gap. If the vendor’s evidence, plus your own testing of the deployed SDK configuration, shows PAD coverage at the assurance level you need, rely on the SDK. If the testing finds no liveness check, no replay protection, or an enclave that only secures keys and communications, add a supplementary PAD layer for exactly the missing checks. Repeat the testing over time: NIST SP 800-63B-4 notes that an authenticator’s ability to resist attacks typically degrades as threats evolve. Turning ad hoc spoof testing into structured evidence is covered in our piece on validating face-detection defences with structured evidence rather than ad hoc pentests.

How do you apply adversarial patches, glare or laser dazzle the same way on every run?

Fix the rig and sweep the variables. Generate the attacks with published methods and existing toolkits, then build a thin physical harness that controls distance, angle, placement and lighting and logs every frame. The hard part is making the physical application reproducible, not inventing new attacks.

TechnoLynx can define the threat set, extend the attack generation and build the physical harness, integrate it with your perception hardware and pipeline, confirm measurement consistency across repeated trials, and hand the protocol over for red-team regression use.

A defensible baseline threat set

A red team reproducing published work against a traffic-sign model has a clear starting set. The original adversarial patch paper produced universal, targeted patches trained with Expectation over Transformation (EOT) so that they survive changes in scene, position and viewing transformation. The RP2 attack used black and white stickers on a real stop sign, causing targeted misclassification in 100% of lab images and 84.8% of video frames captured from a moving vehicle. Its authors proposed a two-stage evaluation of stationary lab tests followed by field tests. For glare, the Adversarial Laser Beam method (CVPR 2021) manipulates a laser beam’s physical parameters to attack a classifier. A survey of traffic-sign attacks adds shadows, raindrops, translucent patches and emoji stickers to the list.

Existing tools generate the artefact, not the physical test

Existing tools generate the artefact. The Adversarial Robustness Toolbox (ART) provides runtime detector interfaces and robustness metrics such as empirical robustness, the average minimal perturbation a successful attack needs. The REAP benchmark ships tooling to render patches onto real sign images with geometric and lighting transforms, built on PyTorch and Kornia and integrated with detectron2. The physical step (printing, projecting, mounting and controlling the environment) is the part you build yourself.

A physical test of naturalistic adversarial patches on a Quanser QCar testbed shows what that step looks like. It swept patch type (three naturalistic adversarial patches plus plain white and black squares), three sizes, three placements on the sign and five camera-to-sign distances. The detector ran for a fixed 15-second window with vehicle and sign stationary, and confidence was recorded per frame. The plain occluders sometimes matched or beat the adversarial patches, so the authors concluded that occlusion baselines are necessary: a confidence drop may come from covering the sign rather than from the attack’s structure. Lighting needs tight control. REAP found the lighting transform affected attack success more than the geometric one, and a large-scale commercial-vehicle test kept every run between 25,000 and 30,000 lux measured with a light meter. Simulation can widen the sweep cheaply: PADetBench uses CARLA to align weather, viewing angle and distance exactly across attacks. Simulated physics may still under- or over-state robustness, so use simulation to search and the physical rig to confirm. Holding results stable as weather, light and camera position vary is its own problem, covered in our article on building scenario-based perception robustness testing as a durable safety gate.

Measure at the system level

Model-level numbers overstate what an attack does to the deployed system. The REAP authors found metrics on a synthetic benchmark could differ from realistic conditions by up to 50 to 60 percentage points on average, always overestimating the attack. A physical test of four commercial vehicles found 6.67% overall transfer success across 30 attack combinations, against 51.6% on average in the original papers. One likely cause was spatial memorisation, which keeps a detected sign in memory until the vehicle passes it. Score the protocol on what the vehicle displays or does, not on per-frame confidence, and version-control the attack set and protocol so that a model update does not silently break comparability.

What misclassification threshold can we defend to a type-approval authority?

UL 4600 and the automated-driving safety frameworks reviewed here publish no robustness threshold to borrow. A defensible threshold is derived from your own hazard analysis and measured error distributions, written down with its reasoning, and owned by your organisation’s safety and cybersecurity sign-off authority. This applies whether the sign-off is under an automotive cybersecurity regime such as UNECE R155 or an internal procurement review.

The frameworks back this approach. UL 4600, published in April 2020, does not define performance or pass/fail criteria for safety, nor acceptable risk levels. A review of safety argumentation frameworks for automated driving found no consensus on a methodology or set of safety metrics. It notes that thresholds and pass-fail criteria are not standardised and recommends weighing technology limits, human driver behaviour, social acceptance and legal aspects when setting targets. Philip Koopman’s 2023 overview of UL 4600’s third edition describes a goal-based safety case of claims, arguments and evidence. Self-certification is permitted with an internal assessor, and safety performance indicators measure whether claims keep holding in operation.

Worked example: three thresholds others derived, and how

Assumptions: each figure below is the published derivation in its own domain, shown as a pattern to copy, not a number to reuse.

1. Biometric PAD, from a standard. FIDO sets IAPAR at or below 15% per PAI species for BioLevel 1 and at or below 7% for BioLevel 2. With 150 attack transactions per species, that means at most 22 errors for 15% and at most 10 for 7%. The arithmetic: 22/150 is 14.7% and 10/150 is 6.7%.
2. Localisation, from physical consequence. The FusionRipper study defined attack success from road geometry: 0.295 m of lateral deviation to touch the lane line on local roads and 0.895 m to leave the road, computed from lane, car and shoulder widths in US Department of Transportation design guidance.
3. GNSS interference monitoring, from a false-alarm budget. The authors note there is no universally recognised jamming threshold. They set a false-positive goal of one in a million, roughly one second of false alarm every 12 days, and estimated the false-positive probability by falsification using fuzzing with importance sampling.

All three follow the same pattern: state the consequence that matters, measure the error distribution on your hardware, and fix an explicit error budget before testing. Threat-model the inference pipeline first, because it tells you which adversarial scenarios are safety-relevant and therefore where the threshold should bite.

A self-derived threshold carries acceptance risk if the authority disagrees with your rationale. Thresholds may also need revising as new attacks are published, and without industry consensus, cross-OEM comparison stays limited.

Can an existing benchmark be extended to spoof our thermal-RGB drone, or does it need its own rig?

Expect to build a bespoke rig around the production airframe and payload. Score the rig on graceful degradation rather than binary pass or fail.

Fusion changes the threat set. An ICCV 2023 study notes that single-modality physical attacks fail against combined visible and infrared detectors, because a visible perturbation is not captured by the infrared sensor and a thermal change does not show in visible light. Its unified patch, cut from aerogel, fooled both modalities at once with a 73.33% attack success rate against YOLOv3. The physical tests give a usable rig template: a person filmed at 4 m for 20 seconds at 10 frames per second, then varied by ±15° and ±30° angles, 6 m distance, a sitting posture and an outdoor scene. Success fell from 73.50% frontally to 57.00% outdoors. The study counts an image as attacked only when both modalities are fooled, which is the right metric for a fused stack.

Graceful degradation needs its own test. A thermal-visual fusion study degraded one modality at evaluation time with a 15×15 Gaussian blur (plus intensity scaling of 0.6 on the visual channel) and found that detection did not collapse. That tests natural degradation, so a spoof rig has to run the same comparison with adversarial inputs. Three measures separate graceful degradation from silent failure: the confidence decay curve, detection latency under attack, and whether a defined fallback mode triggers. Confidence-averaged metrics such as mAP50:95 catch drops that stay above a fixed detection threshold. There is a regulatory analogue for the fallback: the FAA’s airworthiness criteria for one UAS type require a predetermined automatic action on loss of the command-and-control link, documented in the flight manual.

Synthetic spoofing is cheaper and more repeatable but may not capture real optical and thermal physics. Results from one airframe and payload need revalidating on another. Dazzle lasers and decoys bring safety and regulatory obligations that sit outside the technical protocol.

To tell a spoofing attack from smoke, heat haze or RF noise on that airframe, we recommend a dedicated sensor-integrity monitor layered on the degradation monitor, built on thermal-versus-RGB consistency checks, with its own severity-classified alert channel. Measure the monitor’s false-alarm rate under dense smoke that degrades both channels together before trusting its alerts.

The FAA’s airworthiness criteria for one UAS type require security risks to be identified, assessed and mitigated, and set safety outcomes rather than prescriptive tests. The practical evidence pack is a safety case (threat model, mitigations, test evidence), strengthened by independent third-party adversarial testing, in a format agreed with the procuring authority before testing starts.

How do you tell a spoofing attack from ordinary sensor degradation?

Pair the primary signal with a second, physically independent observable that attacks and natural degradation move in different ways, and set the false-alarm budget explicitly. GNSS shows the principle clearly. A GNSS monitoring study notes that C/N0 alone can mislead: obstructions and multipath look like jamming, and a spoofer can balance power to keep C/N0 nominal. Plotting C/N0 against calibrated received power, measured with COTS u-blox F9P receivers costing under $400 each, flags a region where C/N0 is too high for the received power, which suggests spoofed signals. Over an 8-hour test dataset, overall detection accuracy was 99.4%; in separate evaluations, distinguishing spoofing from jamming was significantly less accurate. A power-distortion detector for civil GNSS receivers shows the same split. It flagged every spoofing and jamming attack across more than 25 datasets with a single-channel false-alarm rate below 0.6%, yet labelled spoofing as jamming more than four times in five.

Detecting that something is wrong is easier than saying what caused it. A simulated oilfield study of separating instrument faults, weather effects and data manipulation detected attacks fastest (median 1.7 hours) but attributed them correctly at the moment of alarm only 42% of the time, rising to 73% six hours later. Because degradation calls for maintenance and manipulation calls for incident response, the authors found most of the value lay in deferring ambiguous cases to an analyst.

Other sensors have their own physical invariants. For LiDAR, the CARLO defence checked occlusion patterns and cut attack success from more than 95% to below 9.5%, with anomaly-detection precision of at least 99.5%. A temporal-consistency check that compares detections with motion predicted from earlier frames detected spoofed vehicles more than 98% of the time, with 91% recall and about 5 ms of added overhead at 41 Hz. For camera patches, the APRICOT authors found that physical patches could be flagged as anomalies even without attack-specific training. Production-grade fusion localisation was still spoofable. FusionRipper reached at least 97% success for off-road deviation against production-grade multi-sensor fusion localisation. Camera lane detection is a candidate cross-check, but it is itself vulnerable to physical attacks.

A resistance claim is a dated measurement

Every result above holds for one attack set, one firmware or SDK version, one threshold and one set of conditions, and it expires as attacks improve. NIST’s March 2025 adversarial machine learning taxonomy notes that defences evaluated against weak attackers have repeatedly been broken by stronger ones, and that mitigations must be tested against strong adaptive attacks. So the useful deliverable is not a pass certificate. It is a versioned protocol, a logged dataset and a threshold rationale that someone will rerun when the model, the firmware or the published attack set changes. The question left to settle is who owns that rerun and what triggers it. Our guide to building a defensible, continuous red-teaming and adversarial-testing programme works through that question.

FAQ

What is an adversarial attack?

An adversarial attack crafts an input whose classification can be changed to a class the attacker chooses, often with a perturbation too small for a human to notice. Physically realisable versions, such as printed eyeglass frames or stickers on road signs, carry the attack into the real world, where the perturbation must survive changes in viewing distance, angle, lighting and camera.

How does a liveness check work?

A liveness check, or presentation attack detection, decides whether a captured biometric comes from a live person at the sensor rather than an artificial replica, using information beyond what normal authentication collects. Passive liveness scores a single capture without asking the user to blink or turn their head, and it usually runs before matching so that a spoof is never matched.