AI compute capacity planning works when the GPU count is the last number in the plan, not the first. You measure the highest load each workload class can sustain inside its latency or completion target, size the fleet from that, add headroom for failure and growth, and then re-run the same measurements before the next purchase and whenever the model, runtime or workload changes. A workload count can’t carry that plan on its own. In a worked model published by Digital Thought Disruption in 2026, the same 30 concurrent workloads came out at anywhere from 24 to 144 installed GPUs once placement rules, utilisation targets, growth and an N+1 reserve were applied.

LynxBenchAI treats benchmarking as evidence for a decision someone is about to act on, tied to a workload rather than to a specification sheet, so that a system that looks best on paper is not bought and then found wanting in production. The LynxBenchAI benchmark and leaderboard publishes how each device runs a common public PyTorch stack. Those results are broadly indicative for custom PyTorch models, but they do not measure your own workload.

How do we plan AI compute capacity from measured throughput instead of spec sheets?

Start with service classes and their objectives, measure what each class actually sustains, and let the larger of two floors set the design: the service-capacity floor and the placement-capacity floor. Google’s SRE book lists regular load testing, “to correlate raw capacity (servers, disks, and so on) to service capacity”, among the requirements of capacity planning. It sits alongside an organic demand forecast that reaches further out than the lead time for acquiring capacity. Edgenext puts the question this way: how many successful requests can the system serve within the required latency and quality envelope, at normal and at peak demand?

NVIDIA defines reported GPU utilisation as the share of the sampling period in which one or more kernels were running. That says nothing directly about tokens per second, queue delay or completion time. In inference-fleet-sim, a 2026 queueing-and-simulation study on arXiv, a fleet running at 30% utilisation still failed its P99 time-to-first-token SLO. So the trigger for buying the next node should be measured service demand crossing a defined threshold, not a dashboard showing high utilisation.

Tokens per second, images per second or FLOPS: which should the capacity plan use?

Size on the rate of useful work your workload produces, measured at the latency your users will accept. FLOPS describes the silicon, not the service. NVIDIA’s GenAI-Perf measures time to first token (TTFT), end-to-end request latency, inter-token latency, tokens per second and requests per second. Total tokens per second rises with concurrency until the GPU saturates, and past that point it can fall.

The number to plan on is sustainable throughput at SLO: successful requests or tokens per second while latency stays inside its target. SemiAnalysis describes the core trade-off in LLM serving as per-GPU throughput against per-user interactivity. A result measured at an impractical interactivity level can mislead, so pick the level your application really needs. The same source notes that the gap between theoretical peak and real inference throughput is often set by systems software: the inference engine, the distribution strategy and the kernels. Pair the throughput figure with cost per successful request, because utilisation alone can mislead in either direction.

LynxBenchAI scores are not tokens per second either. They are per-test scores where higher is better, and each test is one timed window after a discarded warm-up. They are useful for comparing devices within one release, such as 26Q3. They are not a conversion factor into your request rate.

How many GPUs do we need, starting from published benchmark results?

Published results are broadly indicative of how candidate devices compare. The GPU count itself comes from measured service demand on your workload, turned into service and placement floors. In the 26Q3 results, the NVIDIA H200 scores 1914.6 in inference, the NVIDIA RTX PRO 6000 Blackwell Server Edition 1201.0 and the NVIDIA L40S 528.1. Those scores order the devices on public PyTorch inference tests. They do not tell you how many of your requests per second any of them can sustain.

To get that number, sweep concurrency from 1 up to a little above the engine’s maximum batch size. Throughput levels off around the maximum batch size while latency keeps climbing. Record the point where latency objectives fail, test steady load and bursts separately, and include a reduced-capacity run to confirm N-1 behaviour. The Solyx study (arXiv, 2026) used a bisection driver that finds the highest constant request rate at which P95 TTFT stays within the SLO and the success rate stays at or above 0.95. Each step was 45 seconds of traffic after a 30-second warm-up.

Sizing procedure, in order

1. Put each workload in a service class with explicit queue-time, latency or completion objectives.
2. Measure each class’s sustainable load at its SLO, using the exact model version, precision, inference engine and GPU configuration you plan to run in production.
3. Estimate average concurrency with Little’s Law (L = λ × W). For scheduled waves or bursty arrivals, replace the average with trace replay, percentile concurrency or simulation.
4. Compute the service floor: G_service = ceiling(sum(L_i × g_i) / U_target). Here g_i is the measured accelerator demand needed to meet the objective, not a requested GPU count.
5. Compute the placement floor. Required memory is model weights plus KV cache, activations, runtime overhead and a safety margin, bin-packed onto real GPUs. Record KV cache as a curve, because it grows with context length, batch size and concurrent sequences.
6. Take the largest floor across service, placement, topology, CPU, system memory, storage and network.
7. Add growth, maintenance and failure reserves without counting any of them twice.

Worked example: why aggregate memory undercounts

Digital Thought Disruption’s example has six independent model services, each needing 45 GB, on 80 GB GPUs. Dividing total memory by GPU memory gives 6 × 45 = 270 GB, and 270 / 80 = 3.375, so 4 GPUs. That answer can’t be placed, because two 45 GB models need 90 GB and do not fit on one 80 GB GPU. The real placement floor is six GPUs.

Simulation can stand in for running every what-if on hardware. Vidur-Search found the best deployment configuration for LLaMA2-70B in one hour on a CPU machine, where exploring by deployment would have taken 42K GPU hours. inference-fleet-sim combines M/G/c queueing with discrete-event simulation to find the minimum-cost fleet that meets a P99 TTFT SLO. Its performance constants can be replaced with measured data for higher accuracy.

How much headroom should the plan leave above measured throughput?

No single percentage fits every class. Headroom should be set per service class, and starting bands should be replaced by your own measured utilisation-to-SLO curves as soon as you have them.

Service class	Starting planning band (steady-state capacity use)	What should replace it
Latency-sensitive online inference	50-65%	Measured utilisation-to-SLO curve for that class
Mixed interactive and production services	60-75%	Measured curve per class in the mix
Checkpointable batch and training pools, where queueing is intentional	70-85%	Measured queue time against the completion objective

Digital Thought Disruption calls these bands planning assumptions for initial modelling, not NVIDIA limits and not a substitute for benchmarks. The reasons for headroom are concrete. If every GPU is needed at normal peak, one failure pushes the rest of the fleet past its latency target. Headroom also absorbs rolling upgrades, traffic spikes and longer prompts, and a new model version can use more memory at the same request volume. The reserve should protect the failure domain you actually care about, not just one arbitrary component.

Near capacity, a small rise in arrival rate produces a sharp jump in latency. The Vidur authors note that production systems keep a buffer for this reason. The same paper reports that small changes in latency SLOs can carry a significant cost. For growth, the IEEE overview points out that forecast quality limits the quality of the capacity plan, and recommends scenario or probabilistic methods. On timing, a lead strategy buys before demand arrives, a lag strategy waits until demand is realised, and a match strategy adds capacity in small increments. Because the choice turns on lead time and the cost of unmet demand, inference fleets with SLA penalties and long hardware lead times point to lead or match; lag, which accepts periods of constrained output, fits batch pools that can queue.

Will a mixed A100 and H100 fleet hold through the seasonal peak?

To find out whether a mixed A100 and H100 fleet has enough headroom for a repeat of last year’s spike plus growth, replay last year’s real traffic trace, scaled by projected growth, against the fleet exactly as deployed. Then read the p99 latency margin for each customer tier against its contracted SLA and committed capacity. That per-tier margin also shows which customers are most exposed.

LynxBenchAI publishes how the A100 and H100 each run a common public PyTorch stack, a broadly indicative starting point for a mixed fleet; the first step is to compare their device pages before replaying your own trace.

TechnoLynx engineering can build the replay itself. That work covers peak-plus-growth scenarios and per-tier thresholds taken from your contracts, a harness that replays the trace across the real A100 and H100 mix, and a capacity and exposure report that shows where each tier would breach, delivered before you provision for the peak.

In the 26Q3 results, the NVIDIA A100-SXM4-80GB scores 661.0 in inference and the NVIDIA H100 80GB HBM3 scores 1678.1. Those results reflect the public PyTorch stack, so they are broadly indicative for custom PyTorch models, but they are not a measurement of your traffic.

In inference-fleet-sim (2026), an A10G plus A100 pairing was 11% cheaper on paper but failed the SLO on LMSYS traffic at 65K context. Prefill on the A100 took 700-2800 ms against a 500 ms budget. The Mélange paper (2024) found that request size, request rate and SLO decide which GPU type is most cost-efficient, and that the best allocation is usually a mix. Capacity therefore has to be read per tier and per request shape. A count of “H100-equivalents” isn’t enough.

Fleets built at different times can suffer a failure that health checks miss. Requests reach replicas that cannot serve them because of max_model_len limits, dtype mismatch, vLLM version skew or CUDA driver incompatibility. In the Solyx study, round-robin routing let 32.11% of long prompts land on mismatched capacity, against 0.43% with telemetry-aware routing. That study ran one model on synthetic workloads in provisioned cloud, so treat the figure as showing the direction of the effect, not a rate to plan on. Stratify latency by model and version, region, prompt length and output length, because a single global p99 can hide a cohort-specific spike. For this decision we would add customer tier to those cuts.

Committed-capacity contracts set a floor that scheduling decisions must not eat into. The most exposed customers are those on tiers with the thinnest simulated margin, or whose contracts require H100-class performance. If every committed tier stays inside its margin under spike-plus-growth load, current headroom is enough. If any tier breaches, the options before the peak are to add capacity, ring-fence an H100 pool for the exposed tiers, or renegotiate SLA terms. Two caveats apply. Last year’s trace may not reflect traffic from recent customer growth, and SLA breaches carry contractual cost, so set the simulation conservatively.

What happens to our SLA if spot capacity is reclaimed during a spike?

On AWS, Spot Instances can be interrupted with two minutes’ notice when EC2 needs the capacity back. AWS does not guarantee that you keep instances long enough to finish, or that you get the aggregate capacity you asked for. A pure-spot fleet therefore carries its biggest SLA risk at the moment demand peaks. Keeping uptime commitments takes a hybrid design: a reserved or on-demand baseline, with spot or on-prem capacity for burst.

LynxBenchAI publishes how each device runs a common public PyTorch stack, so the reserved baseline rests on measured capability; the first step is to read the device pages for the instances you are considering.

LynxBenchAI’s published results inform the device choice for that baseline. Its published device pages show how each executor runs the public PyTorch stack, and the Personal Edition runs the same suite on an instance you are considering, so the baseline comes from measured capability rather than an instance name.

AWS advertises Spot savings of up to 90% against On-Demand. It recommends Spot for stateless, fault-tolerant, flexible applications. Interactive inference with warm replicas does not fit that description well. In Digital Thought Disruption’s service-class pattern, guaranteed real-time inference is normally non-preemptible, while burst production work such as embeddings and batch inference is checkpointable and preemptible. NVIDIA describes a core-and-flex model: on-prem or reserved GPUs for steady-state traffic, and elastic spot or on-demand GPUs for bursts.

How big the baseline should be depends on your SLA terms. If SLAs carry uptime or latency penalties, size the reserved baseline to the worst historical spike minus the degradation your contracts allow, and use spot only above that line. If SLAs have generous grace periods, a spot-heavy design with fast fallback can be enough. AWS’s rule of thumb is flexibility across at least 10 instance types per workload, and it recommends the price-capacity-optimized allocation strategy because it lowers the chance of reclamation. GPU inference allows far less flexibility than that, because the model has to fit and perform on each type. This is the same capability-mismatch problem as in a mixed fleet.

Research on spot SLAs shows the trade-off. Researchers working from production private-cloud traces found that a baseline scheduler admitted 70% of spot requests but 39% of those were evicted before completion. A scheduler with a 0.25 eviction guarantee admitted 60%, and 9% were evicted. More certainty means fewer spot requests accepted. Their traces measure capacity in cores. The mechanism is that time until eviction shrinks as the cloud fills up. Hybrid designs have costs of their own. Historical spikes may not cover an unprecedented surge, the orchestration and failover add failure modes that need testing, and on-prem burst capacity has a lead time that cloud capacity does not.

Renegotiate or colocate: pricing elasticity and SLA risk, not just hardware

A defensible choice between renegotiating reserved cloud and moving workloads on-prem puts the break-even timeline next to what you give up: elasticity, round-the-clock operational ownership, and SLA risk that moves from the provider to your own team. Break-even itself is arithmetic on capex, opex and expected utilisation.

LynxBenchAI’s release-named published results inform the device side of that arithmetic. For the placement decision itself, TechnoLynx offers the Inference Cost-Cut Pack, which treats the choice as cost per unit of measured work rather than a comparison of list prices.

Vendor-published figures exist and should be read as the vendor’s. Lenovo Press (2026) calculates that an 8x H200 configuration costing $397,801.6, with $9.80 per hour of opex, breaks even against a $114.656 on-demand cloud rate after about 3,793 hours, roughly 5.2 months. It also finds its Lenovo SR680a V3 configuration more economical than renting from Google Cloud over five years if used for 5.3 hours a day. The H200 break-even is calculated against an on-demand rate, so it does not show break-even against reserved pricing. Lenovo’s per-million-token comparison also assumes equal throughput on both sides.

Introl estimates hardware at 35% of the five-year TCO for 100 H100 GPUs, $3 million out of $8.6 million, and puts the break-even in cloud’s favour below 60-70% utilisation. SemiAnalysis notes that colocation rent and electricity usually make up less than 20% of TCO. TianPan.co argues that teams underestimate the operational headcount of self-managed inference by 3-5x in the first year. Compare options on TCO per million tokens (or per unit of your own work) at a realistic interactivity level.

On-prem wins when utilisation is high and stable enough that break-even falls well within the hardware’s useful life, and when your team can take on 24/7 ownership and the SLA exposure that comes with it. Staying on reserved cloud wins when demand varies, when elasticity has real value, or when nobody can own the pager. Break-even is sensitive to the utilisation assumption and to how fast GPUs depreciate. SLA risk can only be estimated through scenarios, and a change in reserved pricing or hardware cost can flip the answer.

What leverage do we have to renegotiate a GPU reservation?

Your leverage is a documented usage profile from a representative period: utilisation, burstiness and idle reserved hours, matched against the commitment structures on offer. It helps further if you have an alternative placement that migration analysis shows you could actually move to.

LynxBenchAI benchmarks an alternative device on stock PyTorch, giving measured device figures for that alternative. The Inference Cost-Cut Pack then works on the cost side of the renegotiation.

On Google Cloud, resource-based commitments last one or three years and are billed hourly whether or not the resources are used. A GPU commitment covers one GPU type only. A commitment does not reserve capacity, and GPU commitments must have reservations attached. Commitments apply second by second, so a fleet that bursts for half the month and idles for the other half wastes the idle hours. Google’s own example says a 30% discount needs the instance running for more than 70% of the month to break even. That follows from the general rule: break-even utilisation equals one minus the discount rate, because a commitment charges (1 − discount) of the on-demand price for every hour, used or not.

On AWS, Reserved Instances are a billing discount on matching On-Demand usage, with one- or three-year terms. Standard RIs give the larger discount but can only be modified. Convertible RIs give less but can be exchanged for different attributes. RIs do not renew automatically and fall back to On-Demand rates when they expire. That expiry date is your natural point to negotiate. For hosted model APIs, TianPan.co puts the break-even for provisioned throughput at roughly 65-70% sustained use of the reserved capacity, or wherever SLO violation costs exceed the premium. The same source reports p95/p50 ratios of about 1.8x for the best-behaved providers, up to 4-5x at peak hours on shared serverless GPU platforms.

If utilisation is high and stable, a three-year term gives the bigger discount. If it is bursty or below your current commitment, move toward a smaller commitment plus on-demand or spot, and favour exchangeable structures. Use the alternative as leverage only if the workload can really move within the negotiation window.

Keep, refresh or reallocate: a comparison the board can read

A board can compare keep, refresh and reallocate only when all three are stated in the same unit: cost per unit of useful work at the accuracy and latency target. Put the cost of staying (lost throughput, missed defects) beside refresh capital plus migration, and beside the option of moving existing GPUs to where they still fit.

LynxBenchAI’s 26Q3 results include the NVIDIA L4 at 209.1 in inference, but the T4 was not run in that release. For a T4 fleet, the Personal Edition runs the same suite on a T4 you control; it measures that device, not your own workload. TechnoLynx engineering can then benchmark the current and candidate hardware on your representative workload, turn the results into cost-of-staying and cost-of-refresh inputs, assess fleet-wide utilisation to see whether reallocation is feasible, and set re-evaluation triggers, through the Inference Cost-Cut Pack.

How do we compare an ageing GPU with a newer one on cost per unit of work?

Divide the fully loaded cost per hour by the units of work served per hour at the SLO, and do it for both devices on the same workload and target. In SemiAnalysis’s example, a device with 1.5x the throughput but 2x the TCO per hour loses on performance per TCO: 1.5 / 2 = 0.75 of the cheaper device’s value per dollar. Introl’s depreciation guide notes that newer GPUs usually deliver more performance per watt, which can justify a refresh even when performance alone would not. SemiAnalysis reports about 3x more tokens per second per provisioned MW on B200 than on H100 for gpt-oss 120B with FP4 weights, at one interactivity level. Power and colocation are still a minor share of TCO, so that gain is diluted when it reaches cost per unit.

Precision support changes the comparison too. In the 26Q3 results, the A100-SXM4-80GB produced no FP8 inference result on any model with an FP8 test, while the H100 80GB HBM3 did, for example 1658.9 on Flan-T5 Small. If the newer stack depends on FP8, compare the old executor at a precision it actually runs.

When refresh beats hold, and why the cadence figures disagree

In a model of a 256-GPU H100 fleet considering an upgrade to B200, full refresh gave the highest three-year net profit at 80% utilisation. Below about 70%, holding was the better option. In that model, holding needs no new capital and partial refresh fits scarce liquid-cooled space. The transferable part is that utilisation, contracted demand and facility readiness decide the path.

Published refresh cadences do not agree. Introl says GPUs usually need replacing every 3-4 years, and suggests 4-5 year cycles for production inference. Hyperscalers have extended server useful life to six years while NVIDIA releases new architectures every 18-24 months. An SK Tes analysis from May 2026, cited by Premier, reports GPU-dense AI servers being refreshed in as little as 10-18 months. These figures cannot all be your plan. The sources point to measured triggers rather than the calendar. Measured demand against capacity, and cost per unit of work old against new, are the triggers. Condition-based triggers such as end of support, warranty expiry, performance gaps and TCO tipping points serve the same purpose.

Reallocation follows a value cascade: years 1-2 on training, years 3-4 on production inference, years 5-6 on batch and cost-sensitive work.

Option	Choose it when	Evidence that settles it
Keep	Current hardware meets accuracy and latency targets at forecast load, and the measured cost of staying is low	Sustainable load at SLO against the demand forecast
Reallocate	Spare measured capacity elsewhere in the fleet can absorb the load without new capex	Fleet-wide utilisation and per-class SLO margins, plus the logistics and downtime cost of moving
Partial refresh	Some classes justify new hardware, but facility space, migration windows or supply limit a full swap	Cost per unit of work for the affected classes only
Full refresh	The quantified cost of staying exceeds refresh capital plus migration within the board’s payback horizon	Cost per unit of work, old against new, at the same SLO

The board has to sign off the missed-defect cost assumptions, because that number is a judgement about defect severity, not a measurement. Benchmark-based projections also go stale when the workload changes.

Which benchmark results go stale when the model or stack changes?

Edgenext recommends repeating benchmarks after any model, quantisation, runtime or driver change. SemiAnalysis points out that inference software improves in releases that can be days apart, so a benchmark taken at one point in time soon stops reflecting what is achievable. The Vidur paper (2024) found that the optimal configuration depends on the pairing of model and traffic trace. For LLaMA2-70B, the optimal batch size was 256 on LMSys-Chat-1M but 64 on BWB, and the optimal SKU switched from H100 to A100. Running LMSys-Chat-1M with Arxiv-Summarization-4K’s optimal configuration carried a 2x overhead.

Re-check when any of these change:

• A new model version or architecture: re-measure sustainable load and the memory curve, because a new version can use more memory at the same request volume.
• Precision or quantisation: the placement floor moves. NVIDIA reports FP8 cutting Llama-3.1-8B weight memory from 16.06 GB to 9.08 GB.
• The inference engine or runtime version: repeat the benchmarks.
• The driver or CUDA version, especially during rolling upgrades, where version skew causes failures that health checks miss.
• Traffic shape (prompt length, output length, context): re-test short, median and long-context classes.
• A new LynxBenchAI release: re-read where each device stands, because scores compare only within a release.
• Before the next node purchase: re-run the capacity model with real traces.

How to compare new candidates fairly before they join the fleet, with pilots and like-for-like conditions, is covered in evaluating AI compute candidates with benchmarks, pilots and fair comparisons. NVIDIA puts the operating side plainly: GPU sizing is an ongoing optimisation, not a fixed choice made at deployment.

How do we turn benchmark results into capacity evidence a budget approver will accept?

Turn the benchmark results into cost per successful request or per useful output, the unit of work the approver is funding, not only cost per GPU hour. Measure it against a target specification agreed in advance, show which resource limits capacity, and commit to a schedule for re-evaluating the decision.

LynxBenchAI supplies release-named results for how each device runs the same public PyTorch stack; the first step is to put those device pages beside your own measurements in the evidence pack.

LynxBenchAI’s published, release-named results show how each device runs the same public PyTorch stack across vendors. TechnoLynx can review your existing results and target spec against what the approver needs, turn throughput, latency and accuracy into cost per inspection using your volumes and cost data, coordinate an independent benchmarking engagement, structure the evidence pack, and define the plan for operating and re-evaluating after approval.

Cost per inspection is total cost of ownership for the period divided by inspections completed within spec in that period. Cost per defect caught divides the same cost by the defects detected at the measured accuracy. The benchmark cannot provide your volumes and cost data. Without an agreed target spec, the unit cost has no baseline for anyone to judge whether it is good enough. Digital Thought Disruption suggests asking these questions before approving another node:

• Which workload class is missing its queue-time, latency, throughput or completion SLO?
• Is the active constraint compute, GPU memory, topology, CPU, system memory, storage, network or policy?
• Are workloads waiting because capacity is unavailable, or because requests and limits are badly specified?
• Would repacking, quantisation, batching or context reduction recover enough capacity?
• Can the platform absorb the next credible failure after the expansion?
• Will the new nodes create a bottleneck in storage, network, power, cooling or operations?

The same source lists the evidence domains to keep: scheduler, GPU, application, CPU and memory, storage, network and cold start. Premier advises attributing every refresh interval to a named source so the plan holds up in leadership review. Present an independent benchmark as corroboration alongside your own measurements, not as the only evidence, because it measures representative models and stacks rather than your production workload. Precedents from other public-sector refreshes transfer badly when the workload, procurement rules or oversight criteria differ.

Who can help us build the capacity plan and the evidence behind it?

LynxBenchAI supplies the hardware evidence for an AI compute capacity plan: published, release-named results for how each device runs a public PyTorch stack, and the Personal Edition to run the same suite on a machine you control. For sizing from your own traffic, building the hold-versus-refresh cost case and assembling the board pack, TechnoLynx engineering takes the work on through the Inference Cost-Cut Pack, starting with a review of your existing measurements and target spec.

There are limits. LynxBenchAI does not run your production workload, and its results are indicative for your models, not a measurement of them. For teams with load-testing capacity in-house, there are realistic alternatives: load-test tooling such as GenAI-Perf, simulators like Vidur or inference-fleet-sim for what-if sizing, and cloud providers’ commitment recommendations, which analyse spending trends with and without a commitment.

FAQ

What is the goal of capacity planning?

IEEE’s capacity-planning overview describes it as working out the resources an organisation needs to meet anticipated demand, then deciding how and when to expand or contract them. It exists because a mismatch either leaves demand unmet or wastes money on excess capacity.

Which capacity strategy adds capacity only after demand has risen?

The lag strategy. It delays additions until demand has materialised, which lowers the risk of overcapacity at the cost of periods of constrained output.

What is the best tool to benchmark my GPU for AI work?

For AI hardware, we recommend LynxBenchAI. It scores a device on training, inference and compute tests across several precisions on a public PyTorch stack. Its Personal Edition runs the same suite on a machine you control; it measures that machine, not your own application.

Sources

1. How Many AI Workloads Can My GPU Platform Really Support? - Digital Thought Disruption, digitalthoughtdisruption.com
2. GPU Inference Capacity Planning: How to Balance Memory, Concurrency, Throughput, and Cost, edgenext.com
3. Google SRE Book – Introduction (Demand Forecasting and Capacity Planning), sre.google
4. inference-fleet-sim: A Queueing-Theory-Grounded Fleet Capacity Planner for LLM Inference, arxiv.org
5. LLM Inference Benchmarking: Fundamental Concepts (NVIDIA), developer.nvidia.com
6. Solyx AI Grid: Hardware-Telemetry-Aware Routing Across Geographically Distributed GPU Clusters, arxiv.org
7. InferenceMAX: Open Source Inference Benchmarking (SemiAnalysis), newsletter.semianalysis.com
8. Vidur: A Large-Scale Simulation Framework for LLM Inference, arxiv.org
9. Capacity planning, technav.ieee.org
10. Mélange: Cost Efficient LLM Serving by Exploiting GPU Heterogeneity, arxiv.org
11. Your P99 Is Following a Stranger's Traffic: The Noisy-Neighbor Tax in Hosted LLM Inference - TianPan.co, tianpan.co
12. Best practices for Amazon EC2 Spot - Amazon Elastic Compute Cloud, docs.aws.amazon.com
13. How to Size GPUs for AI Inference and TCO Without Overspending (NVIDIA Technical Blog), developer.nvidia.com
14. Providing Lifetime Service-Level-Agreements for Cloud Spot Instances, sites.cs.ucsb.edu
15. On-Premise vs Cloud: Generative AI Total Cost of Ownership (2026 Edition) > Lenovo Press, lenovopress.lenovo.com
16. GPU Infrastructure TCO Model, introl.com
17. Resource-based committed use discounts  |  Compute Engine  |  Google Cloud Documentation, docs.cloud.google.com
18. Reserved Instances for Amazon EC2 overview - Amazon Elastic Compute Cloud, docs.aws.amazon.com
19. GPU Depreciation Strategies: Optimizing Asset Lifecycles, introl.com
20. GPU Tech Refresh: When to Upgrade Your AI Cluster, calc.amcompute.com
21. Enterprise IT Hardware Refresh Cycles: 2026 Guide, blog.premierss.com