A GPU cloud pricing comparison has to start with the spread. On one day in 2026, Flopper found an on-demand H100 SXM5 80 GB listed by 16 providers, and the most expensive listing cost nearly seven times the cheapest. GPUPerHour, tracking 31 providers, finds that no single provider is cheapest for every GPU, and the cheapest one changes as inventory moves. So there is no single “market rate” to check your own number against. There is a distribution of published list prices, and it only means something once you compare the same GPU, memory, pricing type and region.
LynxBenchAI handles the measuring side of this check. It measures throughput, latency and cost per unit of work across accelerators on one scale, so a team can weigh price against measured throughput instead of the card’s name. The place to start is the published results for the GPUs on your shortlist, on the LynxBenchAI benchmark pages. Those results describe the hardware, not your application, so they feed into your own cost model and do not replace it.
Why does the same H100 cost seven times more at one provider?
Five things drive the spread:
Provider type. SemiAnalysis finds hyperscaler GPU rental prices higher than those of the large and emerging neoclouds, because hyperscalers mainly serve the enterprise market. Part of what an enterprise buys there is the rest of the cloud: SemiAnalysis notes that AWS customers’ data is already in the same place as the compute.
Region and zone. In the SkyPilot study, NVIDIA GPUs were offered in only 16-50% of the 294 zones across the top three clouds, and their prices varied by up to 9.5x on-demand and 10.8x on spot between the most and least expensive zones.
Pricing type. On-demand, spot, marketplace community hosts and reserved commitments carry different terms, and the trackers cited here keep them apart: Flopper keeps all four in separate rows so that an interruptible instance cannot pose as the market rate.
Scope of the price. “Per GPU-hour” does not always buy the same thing. GPU Economy tags some rates as GPU-only (CPU, memory and storage billed separately, as at Google Cloud and Modal) and others as a full runnable instance. Getdeploying notes that Google Cloud prices the GPU, vCPU, RAM and local SSD separately, so an a3-highgpu-8g price is all four added together.
Normalisation. An eight-GPU instance at $24 per hour is $3 per GPU-hour, but the provider still bills $24 for the whole instance. GPU Economy also warns that a median taken across mixed machine sizes moves with the mix of node sizes rather than with what a GPU costs.
Like-for-like checklist before you compare two prices
ComputePrices only compares two rows when every element of this tuple matches: provider, GPU model, pricing type, GPU count, commitment months, variant and region. Used as a checklist, that becomes:
| Check | What goes wrong if it does not match |
|---|---|
| Same GPU model and variant (SXM vs PCIe, memory size) | DigitalOcean’s cheapest H100 PCIe was $4.41 per GPU-hour on the same GPUPerHour snapshot where the cheapest H100 SXM was $2.90 |
| Same pricing type | A spot rate is cheaper because it can be taken back, not because the provider is cheaper |
| Same commitment term | Reserved rates trade a lower price for a minimum term |
| Same scope (instance vs GPU-only) | A GPU-only line item understates what a runnable machine costs |
| Same region | On-demand GPU prices varied by up to 9.5x between zones |
| Price per GPU, not per node | A node price has to be divided by its GPU count |
GPU cloud pricing comparison across hyperscalers and specialist clouds
Treat these figures as dated snapshots with named sources, not a price list. Hyperscaler on-demand rates per GPU-hour, as compiled by Thunder Compute (a vendor that sells against these prices), were:
| AWS | $3.43 (p4de.24xlarge) | $6.88 (p5.48xlarge) |
| Azure | $3.67 (NC24ads A100 v4) | $6.98 (NC40ads H100 v5) |
| Google Cloud | $5.03 (a2-ultragpu-1g) | $10.98 (a3-highgpu-8g, per GPU) |
Source for every row: Thunder Compute, 2026.
That H100 figure for AWS is corroborated independently: Beri’s check on 2 October 2026 found AWS listing p5.48xlarge at $55.04 per instance-hour on-demand, or $6.88 per GPU. Google Cloud applies sustained-use discounts automatically, up to 30% off for VMs that run the entire month.
Specialist clouds and marketplaces sit well below those rates. In October 2026 GPUPerHour’s cheapest secure on-demand prices were $1.08 per GPU-hour for an A100 SXM4 80GB on Vast.ai and $2.90 for an H100 SXM5 on Ori, excluding spot and peer-to-peer listings. The 30-day median of daily H100 SXM catalogue averages was $2.19 per GPU-hour, a figure that includes all pricing types. SemiAnalysis has described Lambda Labs as setting the on-demand market rate for H100 SXM at $2.99 per GPU-hour, with others following its moves.
A low price does not settle the choice. SemiAnalysis notes a long tail of emerging neoclouds without basic SOC 2 or ISO 27001 certification. For a team storing model weights it has paid to train, that can rule a provider out before price is even compared.
How much do spot and reserved discounts save, and what do you give up?
The discounts are large, but spot is paid for with preemption and reserved with a monthly fee owed even for unused resources.
| Discount | Published range | What you give up |
|---|---|---|
| Google Cloud Spot | Up to 91% off on-demand | Preemption at any time; Google recommends spot only for fault-tolerant work |
| Google Cloud resource-based CUD | Up to 37% for 1 year; 55% (70% memory-optimised) for 3 years | A monthly commitment fee owed even if resources go unused |
| Google Cloud flexible CUD | 28% for 1 year, 46% for 3 years on committed hourly spend | A minimum hourly spend for the whole term |
| Azure reserved (H100 v5) | About 35% for 1 year, 55% for 3 years | Commitment to one region and instance type |
| Azure Spot | 70-82% below on-demand | 30-second eviction notice |
| AWS Savings Plans | Lower than on-demand for a 1- or 3-year term | Cannot be cancelled during the term |
| Vast.ai marketplace | Interruptible often 50%+ cheaper; reserved up to 50% off | Interruptible instances may be paused; reserved requires pre-payment |
On Google Cloud the discount types are mutually exclusive, so a resource gets one discount, never a stack of them. Spot VMs there cannot receive committed-use or sustained-use discounts at all.
Interruption is the cost of spot, and it varies by card. Thunder Compute reports AWS H100 spot on p5.48xlarge interrupting under 5% of the time against 15-20% for A100 spot on p4d.24xlarge, with 2 minutes of notice on AWS and 30 seconds on Google Cloud. Beri estimates that a 4-node job checkpointing every 30 minutes loses about 5% of each hour at a 2% per-node eviction rate and about 24% at 10%. That overhead belongs in the price comparison, because a spot rate you lose a quarter of is not the rate on the page.
Thunder Compute argues spot’s advantage has also narrowed. AWS cut H100 on-demand prices by 44% on P5 in June 2025, which Thunder Compute argues compressed the absolute savings from spot. Thunder Compute also sells A100 80GB on-demand from $1.09 per hour against $1.20-1.65 for AWS A100 40GB spot, though that comparison comes from the vendor making it. Thunder Compute’s Azure guide describes Azure Spot as unsuitable for production inference or interactive workloads.
The two SemiAnalysis sources differ on reserved terms. The ClusterMAX piece said most customers no longer sign 1-3 year H100/H200 deals because so many clouds offer the card, while the later shortage analysis says most rental volume is transacted on contracts of six months or longer, with large neoclouds preferring terms of a year or more. We give more weight to the shortage analysis: it reports March 2026 pricing, and SemiAnalysis says its index is built from a monthly survey of 100+ market participants and checked against transaction data. The market moved between the two pieces, which is itself the argument for re-checking at every renewal.
Hidden costs an hourly GPU price comparison misses
A cloud GPU hour is a unit of capacity. It is not a completed training run or an accepted inference response. Flatkey’s breakdown of the real bill is GPU compute plus storage, egress and networking, idle time, restart and failure waste, support and operations overhead.
Four of those terms are worth measuring before you trust any comparison:
- Idle share. Idle waste is 1 minus useful GPU-active time divided by billable running time; a notebook billed for 100 hours that does 32 hours of useful work wastes 68%.
- Retries. If 1,000 accepted outputs take 1,180 attempts after errors, evictions and failed checks, the listed compute cost is multiplied by 1.18.
- Storage. On Vast.ai, storage is billed for as long as an instance exists, including while it is stopped. Azure persistent storage costs roughly $18 per TB per month for hot-tier Blob in East US, billed even when the VM is stopped.
- Egress and bundling. SemiAnalysis reports that for most neoclouds the GPU price bundles CPU, networking, power and local NVMe, ingress and egress are typically not charged, and network storage costs around 6-9 cents per GB-month.
The denominator that makes providers comparable is cost per completed workload: all-in cost divided by completed runs, accepted outputs or tokens. Cost per request also depends on levers below the model layer, such as batching, caching, routing, quantisation and serving configuration.
At what utilisation does a reserved GPU contract beat on-demand?
A reservation beats on-demand only when your measured utilisation is higher than the committed rate divided by the on-demand rate. Below that line you are paying for held hours you do not use. Above it, the commitment wins.
Worked example: break-even utilisation from published H100 rates
These rates were checked by Beri on 2 October 2026:
| Offer | Committed vs on-demand | Break-even utilisation |
|---|---|---|
| Together AI, 91-180 day H100 tier | $3.19 ÷ $3.99 | 80% |
| AWS Capacity Blocks, p5 | rates not stated; Beri’s stated figure | 75% |
| Nebius, advertised 35% commitment discount | 0.65 ÷ 1.00 | 65% |
The arithmetic for the first row is simple: 3.19 / 3.99 ≈ 0.80, so you must use 80% of the held hours before the reservation saves anything. Beri also notes that the AWS figure rises to 87% for blocks bought from 7 October 2026, when the P5 rate was due to rise to $5.970.
The payoff is lopsided. Beri points out that twenty points under break-even costs as much as twenty points over it saves, but the upside stops at full use while the downside runs all the way to an idle cluster. The FinOps Foundation makes the same point about coverage: each unit of commitment beyond the workload’s troughs adds waste when unused, until extra coverage starts reducing total savings. The conservative position, in FinOps terms, is to commit only to the minimum that covers the troughs. Whether guaranteed capacity is worth paying for beyond pure price is a separate question, covered in when reserved GPU capacity beats shared pools.
For pay-per-call APIs against owned compute, a 2025 arXiv cost-benefit study found on-premise LLM deployment viable mainly for organisations processing 50M or more tokens a month, or under strict data-residency rules.
How do I compare my own GPU or inference cost with rental rates?
Convert your own number into the market’s unit first, then judge it against the right part of the distribution. In order:
- Express your cost per GPU per hour, dividing node costs by GPU count, as the trackers do.
- Pick the matching segment: on-demand against on-demand, reserved against reserved, never spot against a guaranteed rate.
- Read the spread, not one number. ComputePrices shows a GPU’s price as a distribution across providers, tiers and regions because the gap between marketplace floor and hyperscaler list price is real.
- Move to cost per completed workload for both sides, with idle time and failure waste included.
- Adjust for throughput where the cards differ. SemiAnalysis prices compute as $/hr per effective PFLOP for training and $/M tokens for inference, combining price with measured throughput. Measured hardware throughput is LynxBenchAI’s territory.
Keep one limit in view throughout: the public trackers cited here report list prices. ComputePrices states that it does not see or estimate what large buyers negotiate. GPU Economy says most large H100 deployments are priced through negotiated, committed-use or enterprise contracts, none of it public, and asks readers to treat its index as what a public rate card says this week rather than the market-clearing price. SemiAnalysis adds that posted prices often adjust only after the contract market has moved. If your negotiated rate sits below the public list prices, that is not necessarily an error. If it sits above the hyperscaler list price, it is worth a hard look.
For owned hardware, rental rates make a useful yardstick. Silicon Data calculates that an H100 rented at $3 per hour continuously costs about $26,000 a year, roughly the purchase price of the GPU before power, cooling, rack space and maintenance. A full owned-versus-rented comparison needs more than that, which is what total cost of ownership modelling for GPU infrastructure works through.
Embedding APIs and GPU marketplace margins: what you can and cannot benchmark
Hosted LLM APIs publish per-token prices; the 2025 arXiv study lists, for example, $1.25 input and $10.00 output per million tokens for both GPT-5 and Gemini 2.5 Pro. For hosted embedding or vector-search services, start any comparison from each vendor’s current pricing page, normalised the same way as above.
Marketplace commission rates appear in the sources here only indirectly. On Vast.ai, hosts set their own prices and the market sets the rate in real time. GPU Economy’s parser for one marketplace feed reads a renter fee each run and computes the price as host price × 1.05 / 24, which implies a 5% renter fee on that platform at that time. That is one data point from one platform at one time. SemiAnalysis describes LeptonAI, a platform layer that rents GPUs from other providers, as adding a couple of cents per GPU-hour on top. Anyone building a pricing model on a “typical” marketplace margin should treat it as an assumption to test, not a benchmark fact.
Rebuilding the cost model at every renewal
A GPU cost model written once goes stale, because the inputs move in both directions. AWS cut P4 and P5 on-demand and Savings Plan prices by up to 45% in June 2025. SemiAnalysis reports that H100 one-year contract pricing then rose almost 40%, from $1.70 per GPU-hour in October 2025 to $2.35 by March 2026. Silicon Data reports H100s at non-hyperscale and marketplace providers widely available for $2-4 per hour by late 2025. And on Google Cloud, a commitment fee is calculated from the on-demand prices on the day the commitment starts and stays the same for the whole term, even if on-demand prices fall.
A model you can recompute at each renewal keeps its inputs separate and dated:
- Pull 90 days of hourly GPU allocation from your scheduler and compute the floor (minimum GPUs busy in every hour) and average utilisation, as Beri recommends.
- Record every quoted rate with the date you checked it; Flatkey’s rule is never to copy old numbers into a business case.
- Recompute break-even utilisation for each offer as committed rate divided by on-demand rate.
- Rerun the totals with lower utilisation, a higher retry rate and a changed model choice before signing.
- Check terms as well as rates. Beri’s advice is to cap the commitment at the floor and the term at six months, with a substitution right to newer silicon and a downward re-open on list price.
The term question has a structural reason behind it. SemiAnalysis notes that GPU performance per dollar improves quickly with each generation, which makes long contracts attractive to providers and short ones attractive to customers.
Who can help us check our GPU spend against the market and act on it?
TechnoLynx audits the production AI serving path behind a GPU bill, so a team knows whether its cost per request is a pricing problem or a serving problem before it renegotiates, reserves or switches provider. The audit looks at batching, caching, routing, quantisation, kernel work and runtime choice, the serving levers below the model layer, and it does this before anyone recommends replacing the model. For the hardware side of the comparison, LynxBenchAI publishes measured results for AI hardware, which give the throughput term a provider’s price sheet leaves out; it measures the hardware, not your workload. TechnoLynx’s wider work for AI infrastructure and SaaS teams is on the AI infrastructure and production SaaS page.
Some needs are a poor fit for this kind of engagement. A team that only wants today’s list prices is better served by a live tracker such as GPUPerHour, ComputePrices or GPU Economy, each of which publishes its methodology. Commitment purchasing is FinOps practice, and the FinOps Foundation’s guidance describes spend-based and resource-based commitment discounts. And if the question is what large buyers actually transact at, SemiAnalysis’s contract index, built from survey and transaction data, targets that market.
