To size an HSM-backed key injection setup for several parallel lines, load-test the provisioning path at its real peak, add one HSM or HA member beyond that for bursts and redundancy, and confirm the HSM is actually the limiter before you buy more of them. AWS’s CloudHSM guidance puts the sizing rule plainly: load test to find the peak you should expect, then add one more HSM for high availability. The sections below cover the capacity planning around that rule: queueing as utilisation climbs, where the real bottleneck sits, and whether owning the HSM pool is the right call.
TechnoLynx treats this as an engineering measurement problem within its AI engineering work for IoT and embedded devices: a load-test harness that runs the station’s real call pattern against the HSM, and a throughput model across lines that shows which stage sets the daily output. We would start by measuring one existing line end to end before modelling the rest, to see whether the station software, the network or the HSM sets its throughput.
How many HSMs or partitions do we need for a units-per-hour target across parallel lines?
Start from the minimum for availability, then size up from measurement. For production CloudHSM clusters, AWS recommends at least two HSMs in different availability zones, and at least three where newly generated keys have to be durable. Above that floor, the count comes from a load test of your own application, because performance varies with configuration, data size and the other load on the client hosts.
Vendor figures are guidance, and AWS and Microsoft themselves recommend load testing. AWS publishes RSA-2048 signing figures of 2,000, 3,000 and 5,000 operations per second across its cluster configurations. Thales states its Luna Network HSM 7 at over 20,000 ECC and 10,000 RSA operations per second, with up to 100 cryptographically isolated partitions per appliance.
When the average demand sits well below stated capacity and the line still queues, we would check serialised calls, key generation inside the HSM, the client host and the fixture before the HSM’s signing rate. The picture changes if the HSM generates and stores a persistent key per device, because persistent key generation is serialised on the HSM and parallelising it yields limited benefit.
Where adding HSMs stops paying
For repetitive operations with keys that rarely change, an HA group scales roughly linearly as members are added, and Thales has run 32-member groups under long-term full-load testing. Thales also warns that adding appliances is “not an infallible recipe for endless performance improvement”: each member performs best at around 30 simultaneous threads, so Thales advises a group balanced to what your application servers can drive. HA also costs throughput on tasks dominated by slow operations and object creation, which Thales says would perform much better without HA. AWS adds two less obvious reasons to scale out: hitting audit log buffering limits while rapidly generating or importing and deleting keys, and running out of session key capacity.
Is it the HSM, the programming fixture, or something else that limits our daily target?
Find the limiter before spending money, because speeding up a process that is not the bottleneck has no effect on system capacity. In a key injection line the candidates are the HSM, the client host running the HSM library, the network between station and HSM, the station software’s call pattern, and the device programmer itself.
Data I/O states its PSV7000 handler throughput at up to 2,000 devices per hour, with up to 112 programming sockets and an upgrade path to cryptographic key injection. That is a vendor ceiling for one handler, and adding HSMs behind it does not raise it.
In a comparison of bottleneck detection methods, the waiting-time or queue-length method failed to find the primary bottleneck in a dynamic system, and only the active period method and the bottleneck walk identified shifting bottlenecks correctly. The same study measures a process’s influence experimentally: change its speed and observe how much the whole system’s speed changes.
Diagnostic checklist for locating the limiter
- Measure throughput, not latency. AWS advises measuring throughput rather than latency when sizing, because parallelising transactions gets the most from an HSM.
- Check the station’s threading. AWS recommends 50 to 100 threads per application, because network round trips are magnified when every operation is serialised.
- Check the client host. The CloudHSM client handles load balancing and failover, and it needs enough compute not to become the bottleneck itself.
- Check how keys are addressed. Cache key handles and leave frequently used keys on the HSM instead of looking them up, unwrapping or importing them per use.
- Check the network. Thales’s Luna guidance recommends Gigabit Ethernet to maximise performance.
- Walk the line. The bottleneck walk uses observations of starved and blocked stations and inventory levels, needs no calculations, and suits the shop floor.
- Find the momentary bottleneck before averaging. Roser and Nakano find it imperative to detect the momentary bottleneck before calculating averages when bottlenecks shift.
How much headroom above the provisioning target is enough?
Enough for the variability your shifts actually show, sized at the limiting stage. For a single server, Kingman’s formula approximates mean waiting time as the product of a variability factor, a utilisation factor and the mean process time. The practical reading is that waiting grows steeply as utilisation approaches full, and grows faster when arrivals and service times are uneven, so a line with bursty stations needs more spare capacity than its average rate suggests.
We would not use a universal percentage. Thales describes the right shape for performance HA groups: enough members for the usual load, plus extra members for periods of high demand, with an additional unit for bursts and redundancy. We would size headroom from measured variability (shift start-up, changeovers, rework batches) rather than a rule of thumb, and size it at the stage the bottleneck analysis identified, not automatically at the HSM.
Will the HSM capacity actually be there when a line needs it?
HSM capacity you can count on is the capacity left when partitions are taken down, not the nominal pool. That distinction matters in two ways.
TechnoLynx brings its GPU Performance Engineering practice to this question, measuring the throughput a provisioning path actually sustains under load rather than its nominal rating; the first step is a peak load test of one line with partitions taken down.
Planning for guaranteed capacity
Microsoft’s Azure Managed HSM guidance is explicit: each instance has three load-balanced partitions, which are occasionally taken down for hardware healing, patching and service updates. Microsoft says it is safe to plan on two partitions being available, and that if you need a guaranteed number of operations per second you should plan with only one.
Planning for capacity loss during a peak
When demand exceeds what the cluster can serve, CloudHSM returns errors saying the HSMs are busy or throttled. Station software should treat those errors as an expected state with retry and back-off, not a crash, so that a short capacity dip slows the line instead of stopping it. Loss of connectivity is a further risk for a cloud-backed design: in Snowball’s case study of a European Matter OEM, cross-border network reliability was inconsistent and any outage could interrupt active batches at several factories at once.
Should we buy more HSM capacity, outsource key injection, or redesign the pipeline first?
We would redesign and tune first, then buy, and treat outsourcing as a separate decision about who should own the infrastructure. AWS’s order is the same: maximise throughput with the configuration practices first, then add HSMs for more. If the bottleneck analysis points at something other than the HSM, more HSMs will not move daily output.
Owning the pool brings more control, but it raises the need for station hardening, access control, audit logs, rework handling and disaster recovery. The Secure by Design Handbook notes that some teams run a simple flow with a secure station, an HSM-backed CA and device secure storage, while others benefit from specialised platforms. Distribution changes the calculation: in the Snowball case, a per-factory approach meant the OEM’s security team travelling on site for new key injection ceremonies for every new product or SoC, which the case calls operationally unworkable. The design it moved to runs a policy-enforcing EdgeHSM appliance in each factory, with device keys generated inside the SoC and only CSRs leaving the device.
NXP describes EdgeLock 2GO as saving OEMs from building and maintaining provisioning infrastructure, with configurations per product, and set up to include contract manufacturers.
How does the cost of our own HSM pool compare with an outsourced key injection service, and can it guarantee throughput at our volume?
What the comparison depends on can be named:
| Factor | Owning the HSM pool | Outsourced injection service |
|---|---|---|
| Throughput ceiling | Set by your load test and HA design | Ask for a contractual units-per-hour figure at your peak, per site |
| Behaviour on outage | Your HA and offline design | Ask what happens to active batches when connectivity drops |
| Operating burden | Ceremonies, audit logs, rework, disaster recovery | Infrastructure build and upkeep move to the provider |
| New product or SoC | Your team runs new ceremonies and updates stations | Ask about lead time per new configuration |
| Certification scope | Yours to obtain and maintain | Ask whether attestation certificates (e.g. Matter) are included |
Which assumptions in the capacity model are riskiest, and how do we stress-test them?
The riskiest assumption is that a vendor benchmark describes your workload. Microsoft measured its Managed HSM figures one operation at a time against a single partition using the same key for every request; a single RSA-2048 key reaches 900 signs per second, but 900 different keys at one transaction per second each will not reach the same throughput. A provisioning flow that touches many keys or objects per unit can sit well below the headline. AWS’s pkpspeed tool likewise runs under ideal conditions, calling the HSM directly without going through PKCS#11, and AWS recommends load testing your own application separately.
The other assumptions we would stress-test before committing to a customer volume are these:
- Key creation per unit. Run the test with persistent key generation or import enabled if the design uses it, since generation serialises on the HSM and Thales found object-creation-heavy tasks slower under HA.
- Minimum partitions available. Rerun the peak test with only one partition available, the planning case Microsoft recommends for guaranteed rates.
- Concurrency from all lines at once. Drive the HSM from every station’s real thread count, not one test client.
- Connectivity loss. Cut the link mid-batch and record what the line does.
- Shifting bottleneck. Repeat the bottleneck walk on more than one shift; the limiter on a changeover-heavy shift may not be the limiter on a steady one.
Which hardware to benchmark, and how to read benchmark numbers fairly, is a measurement question in its own right; the test above is about your line, not about ranking appliances.
Who can build the capacity model and fix the bottleneck on our provisioning lines?
TechnoLynx builds the measurement and modelling side of this for device manufacturers: a load-test harness that drives the HSM with the station’s real call pattern, a throughput model across parallel lines that shows whether the HSM, the client, the network or the fixture sets daily output, and changes to station software where the limiter is the call pattern. On buy versus outsource, we would set out what each option has to deliver at your peak so the decision rests on measured requirements; that work sits within our IoT and embedded device engineering.
We are not an HSM vendor and not a provisioning service, so the appliance, its certifications and any outsourced injection contract come from those suppliers, such as an HSM manufacturer or a service like EdgeLock 2GO. Compliance evidence is also separate work: proving the chain of trust to an auditor is covered in our piece on IoT chain-of-trust baselines, evidence and staleness checks.
If a shift falls short on provisioned units, what keeps us inside the contract?
The fallback has to be designed before the shortfall, as part of the provisioning ceremony’s failure handling. One pattern in the sources is pre-authorised offline operation: in the Snowball case, once an authorisation package reaches the factory appliance, production continues offline for the approved duration, multiple authorisations run in parallel, and audit logs sync when connectivity returns. Within the approved duration, a connectivity failure then does not stop production.
For a shortfall caused by capacity rather than connectivity, we would weigh the spare HA member running hot, an extra shift on the stage that is actually limiting, or rework batches scheduled against headroom rather than against the next day’s target. Which of these keeps you inside a contract depends on how the contract measures delivery, daily, weekly or per shipment, and we would negotiate that term once the model shows how much variability your lines carry.
