AI threat modeling for a deployed inference pipeline takes five moves. First, separate the threats that exist only because there is a model (extraction, poisoning, prompt injection, membership inference) from the conventional flaws that let an attacker reach it. Then map the assets from the gateway down to the weights and training data, rank attack paths by exposure, test tenant, GPU and sandbox isolation under attack, and phase the fixes with every deferral written down. Internal review is where this work starts, not where it ends. SOC 2 auditors need credible evidence that security controls actually work, including retests of corrected findings.

For teams building AI infrastructure and production SaaS, the shared GPU layer needs the closest look, because a container escape on shared GPU infrastructure reaches other customers’ data and models. Published research on AI-as-a-service platforms has shown exactly that path: a malicious model combined with a container escape gave researchers cross-tenant access to other customers’ models.

Which threats are specific to the model, and which are API security you already handle?

Threats to an inference pipeline fall into two groups. Conventional hardware and software vulnerabilities sit in the scaffolding that runs the model. AI-based vulnerabilities let an attacker manipulate the model, steal it or recover its training data. The UK NCSC draws the same line: ML systems face the full range of traditional cyber threats, plus adversarial ML attacks that target how the model itself operates.

TechnoLynx’s Production AI Reliability work feeds threat findings on the model endpoint into the release-readiness review before launch.

The split matters because the two groups chain together. A conventional code-injection bug can give an attacker control of model inputs, and that control then enables a prompt injection that reaches downstream users and agents. Microsoft’s AI/ML threat-modelling guidance makes the reverse point: leaving traditional threats unaddressed helps enable the AI-specific attacks.

Take a claims-triage model that adjuster portals can call. The ordinary API questions still apply: an inference endpoint that outsiders can reach needs authentication, rate limiting and input validation. What changes is that legitimate, authenticated access becomes an attack path in its own right, because many attacks on AI systems begin with ordinary query access to a model API. The AI-specific threats layered on top look like this:

• Model extraction. An attacker clones a deployed model by querying it and training a surrogate on the outputs, with no access to parameters or training data. The surrogate can then be analysed offline for inversion or data-leakage attacks, outside your access controls and usage limits.
• Membership inference and training-data extraction. The training set is not deployed with the model, but it is partly memorised in the weights, so an attacker can probe it indirectly through the model’s outputs. Black-box attacks use only output likelihoods, confidence scores or generated text to separate members from non-members.
• Poisoning. Manipulated training or fine-tuning data corrupts what the model learns, and poisoning can significantly increase the model’s exposure to membership inference.
• Prompt injection. Against LLMs this is the common form of input manipulation, and it can arrive indirectly through retrieval databases, tool responses or other agents.

The NCSC groups these into seven attack classes, from model characterisation and inversion to model hardware attacks. NIST AI 100-2 (2025) gives the matching taxonomy of evasion, poisoning, privacy and prompt-injection attacks. Differentially private training bounds each sample’s influence but often costs accuracy.

How do you map an inference pipeline’s attack surface from the gateway down to the weights and training data?

We would map the surface bottom-up, starting from the assets instead of a list of attacks. An asset-centric method asks defenders to take every vulnerability already in their threat model and work out how far it lets an adversary compromise each AI asset. Defenders then derive which AI attacks those capabilities enable. The method names eight asset types, including inputs (user input, prompt context, RAG augmentations, tool results), datasets, and model parameters (weights, activations and updates). It scales well: defenders reuse the first-phase analysis when they assess each new attack, so they do not need to reanalyse the product every time. Its authors say defenders will benefit from blending it with top-down frameworks.

STRIDE-AI is one such top-down framework. It splits the surface into five layers (user interface, application, model, infrastructure, data sources) and tags data flows with “probabilistic trust boundaries” before overlaying an AI-adapted STRIDE matrix. Tooling lags behind: popular threat-modelling tools such as the Microsoft Threat Modeling Tool and CAIRIS offer no AI-specific taxonomies. In one evaluation, a knowledge-graph-driven tool found 96 threats in a system where experts had found 44, and the 96 included every expert finding.

Attack surface by layer

Layer	AI-specific threats	Conventional weaknesses that open the path
User interface and gateway	Direct prompt injection; extraction by repeated querying	Endpoints without authentication, rate limiting or input validation
Application and plugins	Indirect prompt injection through plugins and tool results	Agents with no network egress controls
Model serving	Model inversion, model stealing, membership inference	Container escape to the host; leftover GPU memory readable by co-resident processes
Infrastructure	Training-data poisoning; supply-chain compromise	Third-party weights loaded without scanning or sandboxing, though they can enable remote code execution
Data sources	Bias injection; adversarial contamination of public corpora	Training data from untrusted or unverified sources

We would rank the paths by exposure before ranking them by depth. Systems reachable from the internet face high threat levels and need timely remediation, while a device unreachable from outside can usually wait for a scheduled fix. Two caveats apply to any map like this. It is only as good as the data-flow documentation behind it, and it is a snapshot that goes stale as the pipeline changes. Starting early still pays off, since mitigations cost less the sooner the threats are understood.

Should customer plugin code run in gVisor or Firecracker, or in a container per tenant?

The answer depends on what plugin code can reach today, and a documented trust boundary only counts once its enforcement has been tested. Containers use OS-level virtualisation and share the host kernel, so an unpatched flaw can become a breakout to host resources.

TechnoLynx gives the plugin system containment that is tested, not assumed: a boundary chosen among gVisor, Firecracker-style microVMs and per-tenant containers for the plugin runtime, hardened on the target hardware and checked by escape attempts. The first step is mapping the plugin execution path and every resource customer code can touch.

Today’s blast radius is whatever the plugin execution context can reach: filesystem paths, the network, credentials, the cloud metadata endpoint and other tenants’ data. Published cases show how this plays out. At Replicate, researchers uploaded a malicious model container, achieved remote code execution, and found that containers in the same Kubernetes pod share a network that an attacker can use against neighbours. On Hugging Face, a pod could query the node’s metadata service and obtain the node’s cluster role. Secure AI development guidelines call for scanning and sandboxing imported third-party weights, which should be treated as untrusted code. OWASP adds the hardening baseline: no host paths, container sockets, metadata services or unnecessary device mounts in model-serving containers. In one healthcare agent architecture, a NetworkPolicy egress allowlist is the control that breaks exfiltration chains, whatever instructions the agent receives.

MicroVM vs container: what each boundary buys

gVisor puts a user-space kernel (the Sentry) between the workload and the host and passes no system call straight through. In a healthcare deployment it confined the blast radius of a manipulated agent to its sandbox. gVisor’s own documentation states that it does not generally protect against hardware side channels. Firecracker takes a different approach: it pairs KVM with a minimal VMM of about 50,000 lines of Rust, 96% fewer than QEMU. Its microVMs boot in as little as 150 ms with about 3 MB of overhead each.

Boundary	What untrusted code touches	Startup and overhead	Side-channel exposure
Container per tenant	Host kernel directly, narrowed by seccomp	Warm starts through Docker exec are fast	Shares host kernel and hardware; weaker boundary than a VM
gVisor	A user-space kernel reimplementing the system API	Not compared in these sources	No general hardware side-channel protection; 71.2% fingerprinting accuracy by a host-side attacker in a 2024 study
Firecracker microVM	A guest kernel behind hardware virtualisation and a minimal VMM	As little as 150 ms boot; secure-container startup of 50-500 ms against over 1,000 ms for classic VMs	73.04% fingerprinting accuracy by a host-side attacker in the same study

Those last figures come from a 2024 study that identified the application inside sandboxes by reading CPU frequency signals from the host. In that study, both gVisor and Firecracker stayed open to this frequency side channel. The decision follows from the review. If tests show plugin code is already contained, we would keep the architecture and add monitoring. If plugins can reach other tenants’ data, host secrets or the kernel, move to microVMs or per-tenant containers in order of the exposure found. Kubernetes guidance also recommends sandboxing wherever workloads run untrusted code. The cost is latency and resources, which can matter for accelerator-bound plugins, and new plugin capabilities can reopen a boundary that tested clean last quarter.

Does namespace separation or the GPU device plugin actually isolate tenants on a shared GPU?

No. Namespace isolation is not enough for multi-tenant GPU scheduling, and the device plugin is part of the trust boundary, not something that enforces it. Device plugins often need root-level access, host mounts and direct driver interaction. One compromised plugin or pod can therefore become a route to privilege escalation, container escape and lateral movement across tenants.

TechnoLynx prepares the independent GPU-isolation test: it ranks side-channel and tenant-isolation risks to scope it, hardens MIG and device-plugin configuration before the firm arrives, then coordinates scope and remediation with it. The first step is mapping the real topology: device plugin, GPU Operator, namespace and network policy, and the vLLM serving stack.

Confidence that isolation holds at the GPU, not only the container

Time-slicing gives no memory or fault isolation between replicas, while MIG provides both at the hardware layer. MIG also adds error isolation and memory-bandwidth QoS, which MPS and CUDA streams lack. The usual recommendation follows: MIG for most enterprise multi-tenant needs, time-slicing only for lower-sensitivity workloads. MIG is still not airtight. A published attack shows that memory barriers issued in one MIG instance change load timing in others, enough to infer a victim’s kernel launch patterns during LLM inference.

Memory residue is the second gap. If GPU memory is not cleared properly when a process ends, the next workload can read leftover data. Trail of Bits’ LeftoverLocals disclosure (January 2024) showed this across process and container boundaries on Apple, Qualcomm, AMD and Imagination GPUs. On an AMD Radeon RX 7900 XT running a 7B model in llama.cpp, the leak came to about 181 MB per query, enough to reconstruct the response. In the same researchers’ experiments, NVIDIA GPUs appeared to clear local memory between kernels already. A multi-tenant GPU security guide notes that zeroing GPU memory between sessions removes a whole class of these attacks. The NCSC’s model hardware attack class includes GPU side-channel attacks and manipulation of the virtualisation code in cloud or Kubernetes deployments.

The third gap is the runtime itself. CVE-2025-23266 in the NVIDIA Container Toolkit carries a CVSS score of 9.0: a three-line Dockerfile uses LD_PRELOAD through an OCI hook to give a container root on the host. It affects Container Toolkit versions up to 1.17.7 and GPU Operator versions up to 25.3.1. A version floor taken from the September 2024 bulletin (Toolkit v1.16.2, GPU Operator v24.6.2) therefore does not clear it. Internet exposure is irrelevant to triage here; the priority is hosts that run untrusted or public images. Wiz’s lesson is to assume a vulnerability and put at least one strong barrier, such as virtualisation, behind the container. For a vLLM-based stack, we would add KV-cache, prompt logs and cached embeddings to the tenant-separation tests. OWASP asks teams to verify that job teardown actually removes prompt logs and cached embeddings. For how these limits shape serving design, see validating batched, multi-tenant serving architectures before cutover.

Independent validation that a customer’s auditors will accept

When the goal is to prove isolation to an outside party, the evidence has to come from an independent firm, because the value lies in its being unbiased and attributable. For a report to work as audit evidence, it has to separate confirmed, exploitable findings from automated noise. On cost, US and Western European firms charged $1,500 to $3,000 per tester-day in 2026. Good firms book four to eight weeks ahead, and rush work inside two weeks carries a 20-50% premium if it can be staffed at all. The same source shows why the day rate matters: $15,000 buys ten days at $1,500 or five days at $3,000.

Testing depth is a choice too. Black-box testing is the most realistic but slow and more likely to miss vulnerabilities, while white-box testing is faster and more thorough but costs more. If the model is distributed as open weights, downstream parties hold the weights and can inspect activations directly. A report scoped only for black-box testing of your API then says little about the real threat model.

Why add an independent pen test to internal review, and how do you scope it?

An internal team tests the system it believes it built, and an outside team tests the one that is actually deployed. Microsoft’s AI Red Team, drawing on more than 100 generative AI products, reports that you do not need gradients to break an AI system, that LLMs amplify existing security risks while adding new ones, and that the human element is crucial. Standard penetration-testing methods have known gaps: automated tools miss novel vulnerabilities such as AI model poisoning and adapt poorly to Kubernetes and serverless environments. We would name those areas explicitly in the scope. Use the internal threat model to rank where the paid days go.

We would put blast radius in scope as well. A container escape on a shared node can expose other tenants’ process memory, filesystem data and GPU state. Dedicating nodes to tenants limits an escape to that node, but the kubelet and API server stay shared and still allow lateral movement. A scoping checklist that covers both levels:

• GPU device plugin and operator privileges, host mounts and driver access.
• MIG versus time-slicing configuration, and memory clearing between jobs.
• Container escape to the host, including the Container Toolkit version in use.
• Pod access to cloud metadata endpoints.
• Shared queues or caches between tenants, such as a Redis instance serving several customers.
• Direct and indirect prompt injection, including through retrieval databases and tool responses.
• Tenant separation of prompt logs, caches and embeddings after teardown.
• Chained scenarios, not single attacks, since attacks can amplify one another.

Keeping that testing continuous rather than annual is its own discipline, covered in building a defensible adversarial-testing programme for deployed AI. A single report is a point-in-time assessment: infrastructure changes and new AI features can expand the attack surface it tested.

Could a compromised partner integration on the shared gateway reach our model?

Yes, if the cluster control plane, network or credentials are shared without per-tenant boundaries. In a shared Kubernetes cluster every tenant uses the same API server, and a tenant with misconfigured RBAC can query other tenants’ workloads and service endpoints through it. The default network is flat too: every pod can reach every other pod unless a NetworkPolicy denies it.

TechnoLynx’s Production AI Reliability work feeds findings on this pivot path into the release-readiness review before the triage endpoint launches.

On a flat network, a compromised integration can scan the subnet, find services belonging to other tenants and try to exploit them. OWASP recommends per-tenant token, request, concurrency and spend limits, with circuit breakers for abnormal cost or latency spikes. It also recommends scoping model-serving credentials to the specific model, endpoint and environment instead of sharing platform-wide credentials.

API7’s ai-rate-limiting plugin enforces token-based limits per period on LLM routes. Deployment detail matters here. In local mode each gateway instance keeps its own counters, so the effective limit is roughly the configured limit multiplied by the number of instances. Redis mode shares one quota across all instances. Setting allow_degradation to false makes the gateway reject requests when Redis is unavailable instead of failing open.

Map how credentials are actually scoped today before choosing between network policy, token scope or full service separation. None of these options removes insider or supply-chain risk inside the partner itself. A sibling article covers threat modelling and attack-surface prioritisation for a multi-tenant LLM inference gateway in more depth.

How do you phase threat-model findings without it looking like you cut corners?

The fastest credible route to a defensible threat model before launch is phasing: externally reachable, highest-risk paths before go-live, then internal and lower-risk paths under a dated, written plan. W3C’s threat modelling guide supports the premise, stating that a threat model does not need to be exhaustive to be useful.

For the sprint backlog, score each finding on realistic exploit likelihood against the engineering cost to fix it, take the likelihood evidence from the independent test, and separate fixes that fit a sprint from those that need architectural change. NIST SP 800-154 recommends scoring and weighting attack vectors and controls against risk appetite and costs for exactly this trade-off.

Severity alone is the wrong filter. CVSS measures a vulnerability’s theoretical severity in isolation, not how likely it is to be exploited. Its Base Score assumes the reasonable worst case across environments. One published framework chains three sources instead. It first filters on CISA’s Known Exploited Vulnerabilities catalogue or an EPSS score of at least 0.088, then applies CVSS of at least 7.0 to decide what is urgent. Tested on 28,377 CVEs, it reached 9.1% efficiency against 0.5% for CVSS alone, an 18-fold gain, while keeping 85.6% coverage of exploited vulnerabilities.

That framework was tested on CVE records. Design and configuration findings have no CVE to filter on, for example a missing per-tenant token limit, a time-sliced GPU shared by mutually untrusted tenants, a plugin with open egress. Those need the threat model’s own exposure ranking. Prompt injection is a structural feature of LLM systems, not a fixable bug.

A phasing procedure that survives an auditor’s questions

1. Split findings into CVE-backed vulnerabilities and design or configuration findings.
2. For CVE-backed findings, filter on KEV membership or EPSS of at least 0.088, then on CVSS of at least 7.0; KEV items at CVSS 7.0 or above are critical, typically fixed within days; EPSS items at that severity are high, typically fixed within two to four weeks.
3. Rank design findings by threat level (external reachability first), likelihood of compromise and impact.
4. Give each finding an explicit response: eliminate, reduce, transfer or accept.
5. For every reduced or transferred threat, record what remains, who is expected to handle it and the assumption that makes the transfer reasonable.
6. Where the calendar allows, have an external reviewer check the severity scores before the sprint is committed.

The written deferral is what separates phasing from corner-cutting: it proves the risk was seen, weighed and scheduled.

Who can deliver a defensible AI threat model before our launch date?

TechnoLynx brings a phased threat model (external paths first, each deferral recorded) into the release-readiness review before a fixed launch date, through its Production AI Reliability work. TechnoLynx also coordinates the scope of the independent penetration test so the paid days land on the ranked risks.

If a team already has AI-literate security engineers and a flexible date, building the model in-house with an asset-centric method and buying only the external test is a reasonable route. A pen-test firm on its own can supply the independent evidence but scopes better with a threat model as input. Whoever you choose, ask how they would test GPU-level isolation instead of inferring it from Kubernetes controls, and whether a retest is in the quote.

Should the pen-testers come back to check remediation, and will that satisfy SOC 2?

Yes to the first, and only partly to the second. An internal sign-off shows the remediation process was completed; a retest of the specific findings by the external team shows the risk actually went down. Auditors look for the same evidence: whether corrected findings were retested, and whether any risks were formally accepted instead. Most reputable firms include one retest of critical and high findings within 30 to 90 days in the base price; some charge separately, typically 10-20% of the engagement fee. Keep the retest scope identical to the original findings, or the before-and-after comparison stops meaning anything.

SOC 2 audits add expectations of their own. The testing scope should match the systems described in the SOC 2 report. It should include APIs, administrative interfaces and authenticated user roles, plus AI or LLM functionality where applicable, not just the public perimeter. New AI features are among the changes that warrant testing beyond an annual schedule. A Type II report covers how controls operated over a period, so one pen test is a single data point inside that window, not the whole evidence trail.

Security assurance and compliance evidence are related deliverables, not one artefact. Scope them together from the start, and plan a mapping step, because auditors judge findings against the control language already on record.

Sensor spoofing: threat modelling a drone’s thermal and RGB perception

For a drone flying disaster-zone missions, the attack vectors that matter fool the perception stack through the physical scene: heat sources against the thermal camera, light and printed patches against the RGB camera, and artefacts that survive fusion of the two. This is the physical-input end of the same inference pipeline. Fake sensor data corrupts perception, and the wrong state estimate carries through into planning and control.

An NDSS 2026 paper found three vulnerabilities in thermal image processing (equalisation, calibration and lensing) that heat sources can trigger, whether naturally present or deliberately placed. Equalisation flaws cut mean average precision by 50% for pedestrian detection and 45% in fusion models. In road tests at up to 40 km/h, the attacks produced pedestrian misdetection rates up to 100% and false obstacles with a 91% success rate, persisting minutes after the attack ended. A disaster zone full of fires and hot debris is a hostile thermal scene before any attacker arrives, so the automotive results transfer in mechanism even though the platform differs. On the RGB side, adversarial stickers are cheap to print, laser pointers and projectors are small enough to mount on a drone, and infrared light is invisible to people yet detected by cameras.

Fusion is not a complete defence. Fusing sensors that measure different physical quantities raises the bar, because an accelerometer detects real drift that spoofed optical-flow input cannot fake. Yet thermal ghost artefacts still achieved up to 98.0% attack success against the DAMSDet RGB-thermal fusion detector.

We would start from a systematisation of sensor spoofing against robotic vehicles. Its action-flow model identifies 44 action flows and 103 attack vectors, of which 26 had been realised. It covers cameras among six mainstream sensors and includes drones as a victim type. To filter that list, we would test each vector against three published requirements for a spoofing attack to work: can the attacker influence the environment, will the sensor accept the spoofed reading as plausible input, and does the vehicle respond in a way that matters?

We would put RF and GPS jamming in a separate threat model. Sensor input spoofing means sustained, direct control over the vehicle, not random noise. We would handle jamming in the drone’s BVLOS safety case and link the two models explicitly. Defences exist: attack-aware thermal processing cut the equalisation mAP drop from 50% to 4% and suppressed calibration and lens artefacts completely in outdoor tests. Function-level monitoring across action flows is another proposed route. Validate any of them by red-teaming the actual rig, not by reading the paper.

Can ISO/SAE 21434 carry a drone sensor-spoofing threat model?

We would adapt ISO/SAE 21434, the automotive cybersecurity-engineering standard, using its threat analysis and risk assessment (TARA) process and lifecycle risk management as the structure.

The standard leaves room for this. It imposes no particular TARA method, only the mandatory steps in its clause 8. Attack feasibility is rated per path on time, expertise, knowledge, opportunity and equipment, and risk treatment means accepting, avoiding, mitigating or transferring each risk. The gap lies in the threat categories. The worked example in one published 21434 study is spoofing a CAN message to switch on a lamp. Those categories have to be written for the rig, and the adapted method remains a credibility argument, not a certification.

Isolation that only exists on the diagram

A MIG partition, a gVisor sandbox, a per-tenant namespace and a fused thermal-RGB detector have one thing in common: each looks like a boundary in an architecture diagram, and each has published evidence of a gap an attacker can use. That is why the method above ends with adversarial testing, not with a document. Every driver upgrade, new tenant, new plugin capability or new sensor firmware is a reason to re-test.

FAQ

What is a trust boundary?

W3C defines a trust boundary as the place where different authorities, trust domains, privilege levels, operational responsibilities or assumptions meet. In an inference pipeline, the edges between a tenant’s container, the GPU device plugin and the host are all trust boundaries.

What are the steps of threat modelling?

The W3C guide describes an iterative loop: model the system, identify stakeholders, identify and evaluate threats, consider responses, publish, and iterate. NIST SP 800-154 condenses this into four steps: characterise the system and data, select attack vectors, characterise controls, and analyse the model.

What is grey-box penetration testing?

Testers work with user credentials and partial knowledge of the infrastructure, which sits between black-box and white-box testing. It is more efficient and cost-effective than black-box testing while avoiding the high cost of white-box testing.