A GPU total cost of ownership analysis and a chargeback pipeline survive an audit for the same reason: every number goes back to metered usage, not to an assumption. The order of work is to profile what workloads actually consume, build the full cost stack, divide it by productive GPU-hours rather than installed ones, allocate it with a documented driver tied to consumption, reconcile the result to the invoice every month, and only then test the break-even point against cloud. The one-off TCO comparison and the ongoing chargeback should be two reports from one dataset. FASAB’s managerial cost accounting standard (SFFAS 4) states the same principle for US federal cost accounting: cost information built for different purposes should come from a common data source, and the reports should reconcile to each other.

The pipeline in six steps

These steps run in order:

1. Profile real usage. Measure allocated, productive and idle GPU time per workload before choosing any cost driver.
2. Build the full cost stack. Include depreciation, power, cooling, facilities, network, storage, software, support, platform labour, maintenance and failure reserves.
3. Normalise to one unit. Express capacity as a cost per productive GPU-hour, then as a cost per successful outcome.
4. Meter and allocate. Attribute that cost to teams with a documented driver, such as GPU-seconds per namespace.
5. Reconcile monthly. Check allocated totals against the invoice and the general ledger, and track accuracy and variance KPIs.
6. Test break-even and sensitivity. Compare against the effective cloud rate, not the advertised one, and show how the answer moves with utilisation.

Profile real usage before picking a cost driver

It is tempting to start by choosing a driver from whatever metric the billing console already shows. That is backwards. An allocation driver should not be chosen because its data is easy to get. It should be chosen because it reasonably reflects who benefited from the cost, and the organisation should be able to explain that link.

A private platform can report several different GPU-hour costs from the same annual expense, because the denominator shrinks at each stage: installed, available, schedulable, allocated, productive and outcome-producing time. Allocated time only shows that a scheduler assigned capacity. It does not show that kernels were doing useful work. Allocated but nonproductive time covers container start-up, model loading, data starvation, failed jobs, excessive retries, or a workload that has silently fallen back to CPU. Unallocated idle time should carry a reason code.

What a total cost of ownership analysis for AI serving has to include

A defensible enterprise AI cost model includes depreciation, software, support, power, cooling, facilities, network, storage, platform labour, maintenance, failure reserves, and the difference between installed, schedulable, allocated and productive GPU time. Digital Thought Disruption set out this list in its July 2026 enterprise FinOps model. A 2025 cost-benefit paper on on-premise LLM deployment groups the same costs into capital expenditure (GPUs, servers, storage, setup, networking), operational expenditure (electricity, cooling, maintenance, personnel, licensing) and scaling costs as the workload grows. In accounting terms, full cost covers the direct and indirect resources that contribute to an output, depreciation included.

The costs GPU TCO models most often leave out

The missing lines are predictable:

• Power and cooling. SLYD’s TCO calculator notes that power consumption can exceed the hardware cost over a three-year period, and that cooling scales with compute density.
• PUE overrun. A gpuaas.com analysis gives the example of a PUE modelled at 1.2 and delivered at 1.5, which raises the power bill by 25%.
• Staff time. Digital Thought Disruption’s model warns that a low staffing assumption only holds if those functions are automated, funded by another team, or covered by a managed service.
• Depreciation and obsolescence. The same gpuaas.com analysis cites a 50% loss of value in two years as an obsolescence risk.
• Egress. GPUPerHour’s 2026 comparison of cloud providers puts egress fees between $0 and more than $0.50/GB, a 127× spread. The same comparison notes that model checkpoints, dataset transfers and inference responses can move hundreds of gigabytes per training run.
• Host, rack and networking. The hourly GPU formula in a 2025 arXiv paper on inference economics leaves out the host server, rack, networking and labour, and covers only the bare GPU.
• Idle and nonproductive capacity. It shrinks the productive denominator rather than appearing as a line item.

The sources disagree on how big the hardware share is. Silicon Analysts puts GPU hardware at 35–50% of three-year TCO. A gpuaas.com blog citing Introl’s April 2026 model puts hardware at 25–35% of the total. Either way, hardware is only part of the total, and that is the point that matters for a model built only from purchase quotes.

The total cost of ownership formula, per GPU-hour

The 2025 inference-economics paper breaks the hourly cost of a single bare-metal GPU into depreciation, power and maintenance. It estimates each one this way:

Depreciation = P / (Y × 8760 × u)
Power        = kW × PUE × E
Maintenance  = P × m / 8760

Here P is the purchase price, Y the depreciation period in years, u the utilisation rate, kW the average power draw, PUE the data centre’s power usage effectiveness, E the electricity price and m the annual maintenance rate. The paper sets u = 1 as a standard baseline and notes that at lower utilisation the effective hourly cost is the base hourly cost divided by u. That divisor carries most of the economics, so take it from measured data and never assume it.

Worked example: one platform, five GPU-hour costs

Digital Thought Disruption’s July 2026 model applies one annual cost to a 32-GPU private platform across successively smaller denominators:

Capacity denominator	Annual GPU-hours	Cost per GPU-hour
Installed calendar capacity	280,320	$4.60
Available after 4% maintenance and failure loss	269,107	$4.79
Schedulable after 12.5% operational reserve	235,469	$5.48
Allocated at 65% of schedulable	153,055	$8.43
Productive at 78% of allocated	119,383	$10.81

The $4.60 figure is not wrong. It is simply not the cost of delivered work, and the $10.81 productive GPU-hour is the better unit for placement analysis and showback. The same logic applies to commitments. If a team buys 100,000 committed hours and uses 70,000, the effective rate is 100,000 / 70,000 ≈ 1.43 times the nominal rate, which is 43% higher before storage and network services are added.

Putting cloud GPU-hours and owned hardware on a per-request basis

To compare owned and rented capacity fairly, both have to be priced in the same unit of measure. The FinOps Foundation’s allocation guide stresses that on-premise and cloud teams need to report in the same terms. The first step is to convert both sides to a cost per productive GPU-hour. The second is to turn execution time into cost: the 2025 inference-economics paper computes workload cost as total hourly GPU cost multiplied by total execution time. For hosted APIs, the 2025 on-premise LLM paper normalises the API bill to the token volume the local hardware could actually produce over the same operating hours.

GPU-hour alone is not enough. The Digital Thought Disruption model treats it as an infrastructure unit, not a universal unit of value, and lists GPU-memory-hours for fractional GPUs, endpoint-hours for always-on serving, input and output tokens, successful requests and completed jobs as separate meters. Its strongest shared metric is cost per successful outcome, where a request that returned HTTP 200 but failed on accuracy or latency does not count as a success. The weights and per-request figures this produces are what our article on modelling cost per request to justify compute and placement choices covers in depth.

The per-unit figure can also invert depending on how capacity is allocated. A July 2026 arXiv case study of one developer using coding agents found that prompt caching brought the realised API cost down to $0.57 per million tokens, below the $2.83 amortised unit cost of a shared on-premise slice. On total cost of ownership, under Taiwan-market parameters and a symmetric labour model, the same study found that on-premise saved 40.1% under shared GPU allocation, while dedicated reservation cost 43.8% more than the cached API. The study is a single non-randomised case, so it shows how much the allocation regime matters, not what your result will be.

How should real cloud-fallback usage be measured from logs instead of assumed?

Where a hybrid design carries a fallback rate (the share of traffic that overflows to a cloud API or burst capacity), that number should be measured. Treat what follows as engineering judgement, not an established standard. The FinOps Foundation recommends sending AI calls through a central proxy, with API keys tied to a specific use case and the input and generated tokens of each call recorded in a database. Digital Thought Disruption argues for one ledger that covers private infrastructure, public cloud and hosted APIs.

With that ledger in place, the fallback rate is a query: tokens or requests served by the fallback path per period, split by tenant and time of day, and priced at the effective rate actually billed. We would put the measured rate into the TCO as a distribution with its peaks, not as one average. If the logs only cover a short window, say so, and run the break-even at the high end of the observed range.

Which allocation method will an internal auditor accept?

An allocation method holds up under audit when each charged figure traces back to who benefited, why the driver was chosen and reliable source data. NGMA’s test is to follow one shared cost from start to finish. Can you say who benefited? Can you explain why the driver was chosen? Can you trace the calculation back to reliable source data? Could someone else reproduce it from the documentation?

LynxBenchAI publishes measured performance of AI hardware, a reproducible reference for per-device throughput when a TCO comparison includes the choice of GPU. It does not meter your own production traffic, which still has to come from your telemetry. The starting point is the published results on LynxBenchAI.

GPU-seconds, request count, or a weighted mix?

Use GPU-time alone when requests on the shared hardware cost roughly the same. Use a weighted mix when they do not. The table sets out the options:

Driver	Fits when	Watch for
GPU-seconds per namespace	Each team owns distinct namespaces and per-request cost is broadly uniform; cost = namespace GPU-seconds / total GPU-seconds × cluster monthly cost	Shared system namespaces collect overhead that must be distributed or absorbed
Request count	Measuring application demand and service capacity	Breaks down when long-context requests share hardware with short classification tasks
Tokens	Hosted or internal model APIs billed per token	Prefix caching processes tokens once but logs them against several requests
Per-request GPU-time share	Billing teams or customers on real consumption; GPU time split across each batch step by token contribution	Needs inference traces, DCGM or NVML metrics and a price joined together
Weighted mix of GPU-time and requests or tokens	Shared hardware serves requests with very different cost profiles	The weights are a modelling choice that must be documented and re-validated

Last9 argues that flat per-token pricing can be off by 10–50× per request once prefill versus decode, batch sharing and cache effects are counted. Adroit’s 2026 field notes say token weighting is almost always more accurate than request weighting, because long requests use far more GPU time.

Reserved capacity needs its own rule. The OpenCost specification charges workloads the greater of requested and used resources wherever an asset has allocation costs, as GPUs do. AWS Split Cost Allocation Data for EKS also uses the greater of reservation and usage, and redistributes unused instance cost to pods in proportion to their share. Cast AI describes requests-based charging as suited to team accountability and usage-based charging as suited to product cost-of-goods reporting.

Where a weighted mix is used, take the weights from cost-per-request modelling and write them into the allocation rule. NGMA points out that a method which was reasonable when designed may stop being reasonable as programmes, staffing or technology change. A change in model mix is one such trigger.

Standards you can cite when challenged

These are the recognised references an auditor can check:

• FinOps Framework, Allocation capability: it requires an allocation strategy, a tagging strategy and a shared-cost strategy, and defines an Allocation Accuracy Index as directly attributed costs divided by total infrastructure costs, × 100.
• FinOps Framework, Invoicing & Chargeback capability: it covers invoice reconciliation, General Ledger postings and chargeback KPIs.
• FinOps shared-cost guidance: it names three split methods (even, fixed proportional and variable proportional) and notes that mature organisations tend toward variable proportionality.
• FOCUS: an open billing-data schema designed to support chargeback, cost allocation, budgeting and forecasting whatever the data source.
• OpenCost specification: a vendor-neutral method for allocating Kubernetes cluster costs, in which workload costs plus idle costs equal asset costs.
• FASAB SFFAS 4: it sets an order of preference for cost assignment: trace directly, then assign on a cause-and-effect basis, then allocate on a reasonable and consistent basis. It also warns that arbitrary allocation bases distort product costs.

Citing a standard shows that the method is recognised in the industry. The pipeline’s own audit trail still has to demonstrate that.

Reconcile metered usage to the bill every month

Usage telemetry and invoices will not line up automatically. Digital Thought Disruption calls for recurring reconciliation of cloud charges, commitment amortisation, private depreciation, software, energy, shared infrastructure and labour. The FinOps Invoicing & Chargeback capability asks practitioners to check that chargeback data matches the invoice plus any added costs, and to track the percentage accuracy of chargebacks and the variance between estimates and actuals. Its General Ledger Recharge Rate divides cloud spend recorded as journals in the ledger by total cloud spend for the month, and can be calculated per cost centre.

Tagging usually falls apart in the label propagation chain. Tags do not apply retroactively, so a resource tagged on the last day of the month is untagged in every earlier day’s report. Adroit notes that DCGM Exporter does not join Kubernetes pod labels onto GPU metrics automatically. A Prometheus relabelling rule that reads labels from kube-state-metrics is needed. Even exact token tracking leaves a gap, because other infrastructure costs feed into inference too. FASAB’s test of reconcilability is that the amount of a difference can be determined and the reasons for it explained.

Run showback before chargeback

A new AI platform should generally start with showback, which shows costs to teams without changing their budgets, so tags and allocation rules can be corrected first. Digital Thought Disruption warns that inaccurate chargeback damages a platform’s credibility. Practitioner sources differ on how long this takes. Adroit suggests four to six weeks of showback and a move to chargeback once tag coverage passes 80%, while Cast AI suggests 60–90 days. We would set the switch by reconciliation results, meaning the variance KPI held inside an agreed band for consecutive months, rather than by a fixed number of weeks.

Who can build the GPU metering side of a chargeback pipeline?

LynxBenchAI publishes measured hardware results for AI infrastructure and production SaaS teams whose TCO includes a GPU choice.

The FinOps capability assigns inputs such as financial policies, depreciation policy and General Ledger requirements to finance and accounting. For Kubernetes cost allocation alone, the OpenCost specification and AWS’s split cost allocation data for EKS already exist. An in-house FinOps practitioner with platform engineers who can maintain the label chain is a realistic route too.

Finding the break-even point between cloud and on-prem AI serving

Break-even depends on utilisation, so the result is only as good as the utilisation input. Digital Thought Disruption’s model splits private cost into fixed and variable parts and solves for the volume at which private and cloud costs are equal:

Break-even productive GPU-hours = Private fixed cost / (Cloud unit cost − Private variable unit cost)

With a modelled private annual cost of $1,290,211 and a private marginal cost of $0.50 per productive GPU-hour, break-even against on-demand cloud at $15.00 comes to 1,290,211 / 14.50 ≈ 88,980 productive GPU-hours. The sample platform’s roughly 119,383 productive hours, the same platform whose productive hour cost $10.81 in the worked example above, clear the on-demand and effective-committed thresholds and fall just short of beating an aggressive cloud commitment at $11.20. Even so, the author says the model does not prove private infrastructure is cheaper in general. Lower volume, a different cloud agreement or weaker internal operations could reverse the result.

Sensitivity analysis is part of the answer itself. In the same model, cost per productive GPU-hour falls from $15.34 at 30% productive share of calendar capacity to $7.08 at 65%, because most private costs are fixed. Analyses quoted by gpuaas.com put ownership break-even at roughly 70% utilisation for B200 clusters (American Compute) and 60–70% across most generations (KAD), shifting with electricity rates, GPU lifespan and reserved cloud pricing. Read those as external reference points, not as your threshold. SLYD’s calculator notes that break-even is an output of your own inputs and that PUE is a site outcome, so neither can be published as a general figure. The last question is whether the workload will still run at that utilisation in 24 months. If the model architecture is likely to change, the case for payback weakens.

One-off TCO versus ongoing chargeback: same meters, different jobs

A one-off total cost of ownership comparison answers a placement question once, while chargeback allocates the same costs every month to the teams that consumed them. FASAB says reports that use different bases should be reconcilable and explain those bases, with cost information traceable back to the original common data source. The FinOps allocation guide makes the practical version of the point: chargeback structures should match the structures used for forecasting and budgeting.

Digital Thought Disruption recommends separate rates for installed, schedulable and productive capacity, for shared versus dedicated GPUs and for development versus production, with enough detail kept for finance and architecture to reproduce each rate. Every internal service sold (full GPU, fractional GPU, model endpoint, embedding service) needs its own meter and an owner. Version that methodology the way you version code, and re-run the TCO from the chargeback ledger when the model mix changes. This gives the break-even analysis a feedback loop instead of a single forecast.

A GPU-hour figure is only useful once your organisation defines which GPU-hour it means. If the business case used the installed figure and the recharge uses the productive one, the gap between them is your platform’s efficiency, and the methodology needs to state that openly.

FAQ

Can indirect costs be easily traced to a cost object?

Not directly. FASAB defines indirect costs as resources used jointly to produce several outputs, which cannot be specifically identified with any one of them, such as facilities, maintenance and utilities. When direct tracing or cause-and-effect assignment is not economically feasible, these costs are allocated on a prorated basis, typically first to segments and then to outputs.