Most accelerator-preparation stories start at the loading dock: the box arrives, someone racks it, installs a driver, and calls the workload “prepared.” That order is backwards. By the time hardware is on the floor, the decision that mattered has already been made — badly. Preparation is not an unboxing task. It is a profiling and readiness exercise that belongs before the purchase order, and its job is to produce evidence, not to install software. The distinction sounds academic until you watch it play out. A team decides on-premise accelerators are cheaper than cloud, buys a couple of high-end GPUs, stands up the driver stack, and only then measures what the workload actually does to the hardware. The measurement comes back at 20–40% sustained utilisation. That is a number they could have known in a week of profiling, for the cost of a few engineering hours instead of a capital commitment. The accelerator now sits mostly idle, and the ownership case that justified it never existed. What “accelerator prepare” actually means in practice The phrase gets read two ways, and only one of them is useful. The naive reading treats preparation as provisioning: get the card, get the driver, get the framework, run something. The expert reading treats preparation as the act of turning a real workload into decision-grade evidence about whether — and on what hardware — that workload should run at all. The divergence point is simple to state. Does your preparation step produce evidence, or does it just install software? If your output at the end of “preparation” is a working nvidia-smi and a model that loads, you have provisioned, not prepared. If your output is a workload profile — request pattern, batch behaviour, memory footprint, sustained utilisation under realistic load — you have prepared, and that profile is exactly what the ownership decision needs. We see this pattern regularly across GPU-infrastructure engagements: the teams that get on-premise economics right are the ones who treated the profiling data as the deliverable, not the running container. Preparation done this way feeds into the cloud-versus-on-premise decision rather than pre-empting it. Preparation done the naive way quietly commits the capital before anyone has looked at a utilisation curve. What workload profiling data should you gather first? Before any provisioning commitment, the profile you build needs to answer questions the hardware cannot answer for you. Peak FLOPS on a spec sheet tells you the ceiling; your workload tells you how close you will ever get to it — and for most inference workloads, the honest answer is “not very.” That gap between advertised peak and sustained achieved throughput is the single most important thing preparation surfaces, and it is the theme running through our note on reading utilisation rather than just peak FLOPs on the DGX Spark. The profile should capture, at minimum, the following, measured against the target accelerator rather than assumed: Request arrival pattern. Steady stream, bursty, or diurnal? A workload that is busy eight hours a day and idle sixteen has a very different ownership case than one saturating the card around the clock. (Observed pattern across engagements; measure your own traffic, do not borrow someone else’s shape.) Batch behaviour. How large can you batch before latency violates your SLA? Batch size is the lever that most directly moves utilisation, and it is workload-specific. Memory footprint under real inputs. Model weights are the easy part; the KV cache, activation buffers, and concurrent-request working set are what actually determine how many streams fit on one card. Sustained utilisation under realistic load. Not a single-shot benchmark — a sustained run against production-like traffic, held long enough to expose thermal throttling and memory-bandwidth ceilings. That last figure is the one everything else feeds. It is the utilisation number the ownership decision hangs on, and it is a benchmark-class measurement only if you actually run it against realistic load — a five-minute smoke test does not count. Which software stack elements must be validated before deployment-ready? Hardware readiness is necessary but not sufficient. A depressing share of “the accelerator is slow” incidents trace back to a stack mismatch that a validation pass would have caught in an afternoon. The stack is a chain, and each link has to match the target silicon. For NVIDIA targets, that chain runs from the GPU driver through the CUDA toolkit and cuDNN, up into the framework build (PyTorch or TensorFlow compiled against the right CUDA version), and often into a serving-time runtime like TensorRT or Triton Inference Server. For AMD targets the analogous chain runs through ROCm and the framework’s ROCm build. The failure mode is almost always a version skew: a framework wheel built against a CUDA version the driver does not support, a cuDNN mismatch that silently falls back to a slower kernel path, or a TensorRT engine compiled for a different compute capability than the card you deployed on. Validation here is concrete. Confirm the driver supports the CUDA (or ROCm) version your framework was built against. Confirm the framework sees the accelerator and dispatches to the accelerated kernels rather than falling back to CPU or a generic path. Confirm any graph-compilation step — torch.compile, a TensorRT engine build, an XLA lowering — completes against the target compute capability and produces the speedup you expect. If your deployment plan involves multiple cards, validate NCCL and the interconnect topology too, because multi-GPU scaling that looks fine on paper often bottlenecks on PCIe or a missing NVLink path. How do power, cooling, and thermal headroom factor in? This is the part teams skip, and it bites hardest on-premise because — unlike a cloud instance — you own the physics. A modern datacentre GPU can draw several hundred watts sustained, and a multi-card node can pull well past a kilowatt under load. Per NVIDIA’s published specifications, the H100 SXM variant is rated at up to 700W TDP; that is not a number your existing office-grade rack and PDU necessarily support. Two failure classes show up. The first is a hard power or PDU ceiling: the node cannot draw what the cards demand, and either it will not boot fully loaded or it trips a breaker under sustained inference. The second is thermal throttling — the card runs, but ambient temperature and airflow are inadequate, so the GPU clocks itself down to stay within its thermal envelope. Throttling is insidious because the accelerator appears to work; it just quietly delivers less than the utilisation you profiled, which corrupts the very number your ownership case depends on. Preparation confirms three things before commitment: that the power delivery (PSU headroom, PDU capacity, circuit rating) supports the fully loaded node, that the cooling holds the cards at their rated clocks under a sustained run rather than a brief burst, and that your sustained-utilisation profile was measured under those real thermal conditions. A utilisation figure gathered on a bench in a cold lab does not transfer to a warm, densely packed rack. What utilisation threshold justifies owning over renting? Here is the decision the whole exercise serves. Owning accelerators makes economic sense only when sustained utilisation is high enough that the amortised capital plus power, cooling, and operations cost per useful hour beats the cloud per-hour rate for equivalent hardware. The break-even is workload- and price-specific, but the shape of it is stable: ownership rewards high, steady utilisation and punishes idle capacity, because you pay for the card whether it computes or not. The trap is that many workloads that feel busy run at 20–40% sustained utilisation once measured — and that band is typically well below where ownership breaks even over a 12–36 month horizon. This is the same cost-per-useful-hour logic that governs cloud selection; we work through the rental side of it in comparing cost per useful FLOP across AWS and Azure GPU offerings. Readiness scoring: is this workload ready for on-premise accelerators? Score each axis before committing capital. This is a rubric, not a benchmark — the thresholds are planning heuristics, not guarantees. Readiness axis Not ready Marginal Ready Sustained utilisation profiled Not measured Measured, 20–40% Measured, sustained above break-even Load pattern Bursty / diurnal, long idle windows Mixed Steady, near round-the-clock Software stack validated Untested against target Loads, no perf check Driver→CUDA/ROCm→framework→runtime confirmed, kernels dispatch accelerated Power delivery Unknown / office-grade Marginal headroom Confirmed for fully loaded node Cooling under sustained load Untested Bench-tested only Held rated clocks in target rack Ownership horizon < 12 months 12–24 months 24–36 months at steady load If you cannot mark most rows “Ready” with measured evidence, the workload is not ready for a capital commitment — it is ready for more profiling, or for the cloud. How does preparation feed the decision rather than pre-empt it? The whole point is sequence. Naive preparation is the decision — you buy, then you install, and the utilisation truth arrives too late to act on. Correct preparation is upstream of the decision: you profile, you validate, you confirm the physics, and you hand a decision-maker a utilisation figure and a readiness score that says, in evidence, whether ownership pays back. This connects directly to how you will run the accelerator once it is in service. The utilisation number is not a one-time gate; it is something you keep measuring, because workloads drift and a card that broke even at deployment can fall below the line as traffic shape changes. That continuous view is the subject of turning GPU utilisation data into cloud-versus-on-premise decisions with observability tooling, and it is why preparation and monitoring are two ends of the same discipline. If you want the full engineering context for this ownership question, our [GPU engineering practice](GPU engineering) treats profiling, stack validation, and utilisation economics as one connected workflow rather than three separate chores. FAQ What does working with accelerator prepare involve in practice? Accelerator preparation is a profiling and readiness exercise done before any purchase or provisioning commitment, not an unboxing task done after the hardware arrives. In practice it means measuring your real workload against the target accelerator — utilisation, load pattern, memory footprint — and validating the software stack and power/cooling assumptions. The output is evidence about whether to own the hardware, not a running container. What workload profiling data should I gather before provisioning an AI accelerator? At minimum: the request arrival pattern (steady, bursty, or diurnal), batch behaviour against your latency SLA, memory footprint under real inputs including the KV cache and concurrent working set, and — most importantly — sustained utilisation under realistic, production-like load held long enough to expose throttling. The sustained-utilisation figure is the one every downstream decision depends on. Which software stack elements must be validated before an accelerator is deployment-ready? The full chain against the target silicon: GPU driver, CUDA (or ROCm) toolkit, cuDNN, the framework build (PyTorch/TensorFlow compiled against the right CUDA version), and any serving runtime such as TensorRT or Triton. Confirm the driver supports the framework’s CUDA version, that kernels dispatch to the accelerated path rather than a CPU fallback, and that any graph-compilation step targets the correct compute capability. For multi-GPU plans, validate NCCL and the interconnect topology too. How do power, cooling, and thermal headroom factor into on-premise accelerator preparation? On-premise, you own the physics. A datacentre GPU can draw several hundred watts sustained (the H100 SXM is rated up to 700W TDP per NVIDIA’s specs), so confirm your PSU, PDU, and circuit support a fully loaded node. Confirm cooling holds the cards at rated clocks under a sustained run, because thermal throttling silently reduces the utilisation you profiled — and corrupts the number your ownership case relies on. What utilisation threshold should my prepared workload hit to justify owning accelerators over renting cloud GPUs? Ownership breaks even only when sustained utilisation is high enough that amortised capital plus power, cooling, and operations cost per useful hour beats the equivalent cloud per-hour rate. The exact threshold is workload- and price-specific, but many workloads that feel busy measure at 20–40% sustained utilisation — typically well below break-even over a 12–36 month horizon. How does accelerator preparation feed into the cloud-vs-on-premise decision rather than pre-empt it? Correct preparation is upstream of the decision: you profile, validate the stack, and confirm the physics, then hand a decision-maker a utilisation figure and readiness score. Naive preparation is the decision — buying first means the utilisation truth arrives too late to act on. Preparation should produce the evidence the decision needs, not commit the capital before that evidence exists. What does a practical accelerator readiness checklist look like before committing capital? Score six axes with measured evidence: sustained utilisation profiled, load pattern, software stack validated end to end, power delivery for a fully loaded node, cooling under sustained load, and the ownership horizon at steady load. If you cannot mark most rows “ready” with real measurements, the workload is not ready for a capital commitment — it is ready for more profiling, or for the cloud. The GPU Performance Audit is where this begins: it produces the workload profiling data that a readiness checklist turns into a deployment-ready decision. Skip the audit and you are not preparing an accelerator — you are gambling on a utilisation number you never measured.
