A case for AI compute capital is strongest as an evidence package, not as one ROI figure. The package needs four things. First, a five-year total cost of ownership for every option, normalised to the same workload, with utilisation, useful life and residual value written down as explicit assumptions. Second, a sensitivity range on each of those assumptions. Third, a weighted score whose weights were fixed before any vendor was contacted. Fourth, an independent check of the pilot results the decision rests on. Each decision (hardware choice, capacity sizing, phased upgrade, vendor selection) needs its own version of the package rather than one generic justification.
What finance and procurement need to see before they sign
The “AI ROI” line on a board slide is only as good as the assumptions beneath it, so the package has to put those assumptions where a reviewer can find them. LynxBenchAI treats benchmarking as evidence for a decision someone is about to act on, tied to a workload rather than a specification sheet, and the same standard applies to every line below.
| Component | What it must state | Why a reviewer asks for it |
|---|---|---|
| Normalised TCO per option | Capex, power, cooling, facility, staff, maintenance, refresh and residual value over the same horizon and workload | Cloud and on-prem cost structures differ line by line, so a comparison on purchase price alone is not like for like |
| Break-even analysis | Which of the three buy-versus-rent comparisons it answers, and the average utilisation assumed over the whole life | A reservation already prices in much of the saving from owning |
| Assumption register | Useful life, cost of capital, the source of the residual value, the date prices were read, and what is excluded | Each model states its own exclusions, and list prices change |
| Sensitivity ranges | Utilisation, residual value, cloud rate, useful life | Utilisation is the largest variable, and residual value the second-largest controllable one |
| Weighted score | Criteria and relative weights fixed before scoring, plus sensitivity analyses | Criteria and their relative importance have to be stated in advance and applied only as stated |
| Decision brief | Why the chosen option won and why the runner-up did not | It creates an audit trail and prevents the decision being reversed after the fact |
| Independent pilot evaluation | A success criterion set in advance, judged by someone with no stake in the result | Independence has a technical part and a managerial part |
Break-even utilisation settles most buy-versus-rent arguments
Break-even utilisation is the average utilisation at which owning a GPU costs the same per useful hour as renting one on demand. It equals the owned fixed cost per hour divided by the on-demand price.
Worked example: an owned H100 against on-demand rental
Safeguard’s accelerator TCO analysis uses an illustrative all-in capex of $30,000 per H100, a grid point rather than a market price, with a four-year life and an 8% cost of capital. On those assumptions the fixed cost is about $1.72 per GPU-hour, and against the AWS on-demand price of $6.88, owning wins above 25% average utilisation. The arithmetic is 1.72 ÷ 6.88 = 0.25. The prices were list prices read on 25 October 2026, and they exclude taxes, storage, egress and support.
The threshold rises to about 37% at the top of the same analysis’s capex grid, $50,000 per GPU. The harder problem is that the utilisation figure is an average over four years. That average includes the months of procurement, installation and burn-in before the first job runs, the weeks between projects, and the nights and weekends of an inference service. For that reason we would not accept a break-even claim in a business case unless a written utilisation profile sits behind it.
Reservations and commitments change the question
Buy versus rent is really three separate questions: buy versus on demand, buy versus a reservation, and reservation versus on demand. With a reservation or a commitment you pay for idle hours either way. Utilisation therefore drops out of the comparison, and the question becomes whether you can run the hardware more cheaply than the provider can. Google Cloud’s documentation is explicit on this: committed resources are billed monthly until the end of the term whether or not they are used, and a commitment cannot be cancelled once bought. Comparing a commitment with on demand uses the same formula, with the committed price in place of the owned fixed cost. In Safeguard’s example, a three-year commitment at 45% of the on-demand price beats on demand once you use the committed capacity more than 45% of the time.
A Lenovo on-premise versus cloud TCO paper expresses the break-even threshold in hours of use per day. For one server configuration, owning paid off above about 5 hours of use a day against on-demand pricing, 6.17 hours against a one-year savings plan, and about 9 hours against a three-year plan. In other words, discounted cloud pricing raises the bar that owned hardware has to clear. The paper deliberately leaves out networking, facility overhead and routine IT operations to keep the comparison clear. Any business case that borrows its thresholds has to carry that exclusion forward.
How do we size owned GPU capacity while our GenAI pilots are still unproven?
Own only the load you can already describe from historical data, and keep pilot and burst demand on rented capacity until it has a stable pattern. This section takes three questions: how to forecast, whether a hybrid model works, and what bursting arrangements exist. All three have one answer. Structure capacity so that the forecast you cannot trust is never the one you turn into capital spend.
For the forecast itself, we would use ranges or scenarios rather than a point estimate, and size ownership to the cautious end of the baseline range. FinOps guidance recommends forecasting AI costs weekly or monthly because they vary so much. It also recommends tracking cost per unit of work alongside cost per GPU-hour. A frequent cadence is what shows when a pilot has stopped being a pilot.
Owning the baseline and renting the burst
Safeguard’s analysis notes that inference demand follows a daily cycle, so the cheapest design often owns the base load and rents the peak. Where to draw the line reuses the break-even ratio from the worked example. The k-th owned accelerator is busy only during the hours when demand exceeds k. The best owned capacity is therefore the demand level that is exceeded for exactly F/p of the hours (owned fixed cost over on-demand price), and everything above that level is rented. We would draw three layers: own the baseline you are confident in, commit the middle layer where usage is steady but the hardware choice is not settled, and rent the burst.
The rented layers come in several forms, and each carries a different liability for idle hours:
- On demand: costs scale with usage, which suits short-term or bursty work but becomes expensive for sustained operation.
- Reserved instances and committed-use discounts: AWS lists average Reserved Instance discounts of 40% for a one-year term and 60% for three years, and positions Standard Reserved Instances for steady-state usage. Google Cloud bills committed resources whether used or not.
- Amazon EC2 Capacity Blocks for ML: the price depends on supply and demand at the time of purchase, is charged up front, and does not change afterwards. Unused time inside the block carries no extra charge. That caps the pilot’s liability at the size of the block.
- Provisioned API capacity: availability is guaranteed and the rate per token is lower, but the cost per token actually processed can end up higher than on demand, depending on utilisation.
The hybrid model has its own costs. Deciding where each workload runs and moving data between on-prem and cloud adds operational work, and that has to be weighed against the capacity risk it removes. For large deployments, burst capacity can be scarce: the FinOps Foundation notes that GPU resources for large deployments remain constrained, which pushes buyers towards capacity guarantees. Data residency, egress cost or latency limits may rule out bursting some workloads entirely. The structure manages mis-sizing risk without removing it. Ownership of a pilot’s load should be revisited once that pilot shows consistent utilisation over several months, using the break-even comparison above.
How should an on-prem vs cloud TCO model be documented so it survives procurement and audit?
A five-year TCO survives review when every option runs on the same workload, duty cycle and horizon, and when each option carries capex together with power, cooling, facility, staff, maintenance, refresh and residual value. The board can then compare total cost rather than purchase price. Lenovo’s component table lists power, cooling, staffing and facility overhead as on-prem operating costs, along with a three-to-five-year refresh cycle. On the cloud side it lists subscription, storage and bandwidth. Safeguard’s model splits the owned fixed hourly cost into capital, energy, facility and staff. Its base case puts the non-capital part at about $0.48 per GPU-hour. Capital recovery and maintenance add about $0.41 per hour for every $10,000 of all-in capex over four years at 8%.
What a model leaves out matters as much as what it puts in. SLYD’s GPU TCO calculator, for example, states that it assumes zero residual value, no financing and no tax. That makes it a cash-cost model, and it says plainly that the result will not match the net-present-value model a finance team builds. Disclosure of that kind is what lets a model pass audit: the reviewer can see exactly where it stops. Useful life needs the same treatment. In its 2024 Form 10-K, Amazon disclosed that it had moved a subset of its servers and networking equipment from a six-year to a five-year life from 1 January 2025. Safeguard concludes that three to five years is the realistic window for a buyer without a large reuse pipeline.
Suppose the board’s question is which allocation is lowest-risk within a fixed capital ceiling. Run the same model and workload assumptions across every candidate configuration. Rank the candidates on total cost and on named risks: schedule, vendor lock-in and utilisation uncertainty. Then stage a pilot before committing the full amount, and update the model with the power and throughput the pilot measured wherever they differ materially from vendor estimates. A pilot validates only the workload and configuration it tested. It says nothing about energy prices or refresh timing in year four.
Residual value: pin it to market data, in two scenarios
Mercatus’s H100 resale analysis shows how much this one assumption carries. Take an H100 at $33,000 all-in, held for three years at 70% utilisation, which gives 18,396 useful GPU-hours. A 40% residual value gives $1.08 per GPU-hour of depreciation, 25% gives $1.35 and 15% gives $1.52, a 40% swing from a single input. The analysis recommends pulling secondary-market prices for the card and age being modelled, and running both a fast and a slow Blackwell-ramp scenario. At 36 months, H100 residual value ranges from 40% to 75%, depending on how quickly Blackwell becomes widely available.
The gap between book value and market value at the refresh date needs its own budget line. CHG-MERIDIAN estimates a B300 server at about 33% of its purchase price after two years, while a six-year straight-line schedule would still carry it at about 67%. A fair market value lease moves that resale risk to the lessor. CHG-MERIDIAN suggests this fits refresh plans of two to four years.
How sensitive is the case to hardware choice, reserved discounts and placement?
Utilisation matters most. In Safeguard’s H100 base case, raising utilisation from 40% to 80% halves the cost per useful hour, from $4.30 to $2.15. Raising electricity from 5 to 20 cents per kWh changes the fixed cost by only about 10%. Reserved discounts move the cloud side of the comparison, and every discount step raises the utilisation owned hardware has to reach. Cloud list prices also change during a five-year horizon: in June 2025, AWS announced reductions of up to 45% on its P4 and P5 NVIDIA GPU instances, covering both on-demand and Savings Plan pricing. Hardware choice enters through two terms, capex and work done per hour. A device that completes more of your workload per hour can lower the cost per unit of work despite a higher price, so the sensitivity table needs a measured throughput term, not a datasheet figure.
| Variable | Range to test | Where the range comes from |
|---|---|---|
| Average utilisation over life | Include procurement, burn-in, gaps between projects and off-peak hours | Safeguard’s analysis of hidden utilisation |
| Residual value | Fast and slow next-generation scenarios, using secondary-market prices | Mercatus resale data |
| Cloud rate | On demand, one-year and three-year terms, plus a price-cut case | AWS Reserved Instance terms and the 2025 GPU price cut |
| Useful life | Three to five years for a buyer without a reuse pipeline | Safeguard, citing Amazon’s disclosed change |
| Electricity and PUE | Your site’s figures, not published ones | SLYD: break-even and PUE are site outcomes |
Phased multi-year upgrade or replace everything now?
Phasing usually costs more in total than a single project, so the case for it is lower risk and less disruption, not lower cost. The comparison belongs on one page with three things on it: upfront cost, the exposure carried by sites or clusters left on old hardware, and a year-by-year capital schedule for each path. That lets finance and the CTO see the trade-off instead of a yes-or-no choice.
TechnoLynx can build that single-page comparison. It ranks sites by audited capability gap, costs a phased three-year schedule against full replacement, and checks every figure against the per-site audit before finance and the CTO sign.
Guidance on phased industrial upgrades costs each phase at its full installed cost. It adds planned downtime (lost contribution plus fixed costs that keep running), temporary measures and a contingency. Downtime cost is stoppage hours multiplied by the sum of lost contribution per hour and continuing fixed cost per hour. The guidance’s examples are production machines, but the mechanism carries over to compute. A cluster migration has its own stoppage hours, and a phase priced without them looks cheaper than it is. Each phase also needs a defined KPI, a measurement period and a go/no-go rule. Phasing should stop when the measured result already meets the target, or when updated data no longer supports the next phase. For compute, the result measured between phases should be the throughput and power the first phase actually delivered on your workload, not the vendor’s projection.
If this cycle’s budget is tight, upgrade the weakest sites first and schedule the rest over the following years. If the budget allows and tolerance for interim exposure is low, replace everything now. In both cases, we would state the exposure on each deferred site for the next cycle as a capability gap and an outage risk.
Refresh timing is a variable in its own right, not a fixed calendar. A UIUC and Microsoft Azure lifecycle study argues that fixed refresh intervals are no longer enough for AI hardware. It identifies three cases: upgrade early when the new generation is much better, extend the life of existing hardware when it stays competitive, and skip a generation when the gain is marginal. Horizon’s refresh guidance adds that the value left in the existing asset belongs in the business case before the refresh starts. That value may come from continued use, internal redeployment or resale.
How do we score AI hardware vendors on accuracy, throughput and total cost, not sticker price?
We would rate every candidate on the same criteria: accuracy, throughput or latency, total cost and regulatory evidence. Each rating is multiplied by a weight agreed before any vendor is scored, and the results are summed into one comparable number per candidate.
LynxBenchAI supplies the throughput term for the hardware in that matrix. Its published results show how each device runs the public PyTorch stack your own models run on, so they are broadly indicative for custom PyTorch models. The committee can read the published 26Q3 results for 48 devices before the first vendor quote arrives. LynxBenchAI does not run or measure your own workload. Where the choice is between models rather than devices, TechnoLynx runs the comparison as the LLM Selection Pack.
The arithmetic is a weighted sum. A 2025 arXiv paper on low-code platform selection defines the total score as the sum, across criteria, of each criterion’s score multiplied by its weight, with the weights summing to one. The paper sets the weights through stakeholder consultation, priority workshops and constraint analysis, then normalises them. It expands the matrix with sensitivity analyses to produce an auditable decision trail. The paper’s domain is software platforms, but the method does not depend on domain.
Fixing the weights first is what makes the result defensible. The US Department of Energy’s acquisition guide requires evaluation criteria and their relative importance to be stated in the solicitation, with proposals evaluated only on those criteria. It also requires that cost and quality always be included. Alice Labs’ vendor-selection framework makes the same point for AI tools: pre-set weighted criteria reduce cognitive bias.
The weight on each workload category matters because published measurements do not rank devices the same way in every category. In the 26Q3 results, the NVIDIA H200 scores 1914.6 in inference, against 1678.1 for the NVIDIA H100 80GB HBM3 and 1304.0 for the AMD Instinct MI300X. In compute the order changes: the MI300X scores 1023.3, the H100 80GB HBM3 487.1 and the H200 480.8. In training the 26Q3 order is H200 at 812.0, MI300X at 727.2 and H100 80GB HBM3 at 709.3. A matrix weighted towards inference or compute will rank these three devices differently from one that uses the overall GT figure. GT is also an ordinal aggregate, so a GT gap should never be read as a percentage.
Precision support works as a gate before it becomes a score. In the 26Q3 results, the NVIDIA A100-SXM4-80GB produced no fp8 inference result on any model (shown as 0.0). A plan that depends on fp8 therefore has nothing to score on that device for that axis. Each 26Q3 test is one timed window after a discarded warm-up, and scores should be compared within 26Q3 only. A scorecard re-run next budget cycle should not mix figures from different major releases. If your device is not in the published set, LynxBenchAI benchmarks it on your own hardware with stock PyTorch. Whether a vendor’s own published figures carry over to your workload is a separate check, covered in justifying AI compute choices from benchmarks to procurement evidence.
Turning accuracy-per-dollar into a number finance will sign
A finance sign-off needs one headline number. For that number we would use cost per unit of work: the measured cost of processing a fixed volume of your task at a stated quality threshold, with the method written down. The FinOps Foundation recommends tracking cost per unit of work, such as cost per 100,000 words, alongside cost per GPU-hour. Present that ratio next to the composite score, not in place of it. A single cost-quality ratio can hide a latency or accuracy shortfall that matters in operation. Building a full per-request cost model, with batching and growth projections, is a separate exercise. Here it only needs a documented method that finance can re-run.
Putting validation burden and indirect costs into the capital request
A capital request should show the purchase price beside indirect costs such as integration, retraining, system conversion and power. The lowest unit price often turns out not to be the cheapest decision. TechnoLynx’s LLM Selection Pack runs this kind of scored vendor comparison for model and vendor choices.
The Department of Energy guide says cost evaluation can include re-training, system or software conversion, power consumption and life-cycle costs. It also says the solicitation should name the costs that will be considered. Alice Labs reports, from its own enterprise implementations, that the total cost of ownership of AI solutions typically runs two to four times the initial licensing fee once compute, integration, change management and retraining are included. It flags compute overage pricing as a common surprise, because base licence fees rarely cover peak inference costs.
Regulatory evidence is hard to rate on a one-to-five scale. We would treat it as a minimum threshold a candidate must pass before weighted scoring begins, and use estimated validation hours as the proxy where a number is needed. The Department of Energy guide calls this a decision rule, for example “if the management factor is rated less than satisfactory, then the entire proposal is unacceptable”. The matrix needs revisiting when requirements change, and the low-code framework paper notes that weights need periodic recalibration as priorities shift. The final output is a decision brief that combines pilot data, risk findings and the TCO analysis, and states why the runner-up was not chosen.
Should a third party audit AI pilot results before the rollout payment is released?
Yes, if the party that delivered the pilot also wrote and ran its acceptance tests. A consultancy grading its own work cannot give sign-off the independence it needs, and the payment gate is where that independence counts.
TechnoLynx builds the independent side of that gate under AI Governance and Trust. That means defining the accuracy and safety criteria in advance, building an evaluation harness separate from the vendor’s own test suite, and comparing the independent results with the vendor-reported pilot figures.
The HHS guide to independent verification and validation uses IEEE Std 1012-2004 to define what independence means. Technical independence means the assessors judge the work separately from those who built it. Managerial independence means responsibility for the evaluation sits with an organisation separate from the one building or implementing the product. One pilot-design guide turns this into a structure:
- Write one falsifiable success criterion as a single sentence.
- Get procurement, operations, the second line and the shortlisted vendors to agree that criterion before the pilot starts.
- Run every vendor against the same fixed test dataset, representative of the production workload.
- Have a second-line or compliance team evaluate the results: not the team that selected the vendors, and ideally not the team that will operate the system. Treat the criterion as a binary gate.
- Do not renegotiate the criterion afterwards when a vendor narrowly misses it.
The harness should be sized to the stakes. A clinical or financial decision-support system warrants a deeper evaluation than an internal search tool. Three caveats belong in the plan. The audit adds time and cost to the milestone schedule. Its quality depends on a representative test dataset rather than vendor-supplied cases. And its independence can still be challenged if the auditor has any prior relationship with the vendor. Whether pilot results hold at full production scale is a separate question from whether they were graded honestly, and it needs its own check.
Who can run an independent evaluation of our AI pilot before we pay for rollout?
TechnoLynx runs independent evaluations of AI pilot results under AI Governance and Trust for buyers who need sign-off evidence the delivering vendor did not produce. On whether to bring in a third party at all, the work starts by defining the accuracy and safety criteria the independent evaluation must test. On what the harness looks like, TechnoLynx designs it separately from the vendor’s own test suite and runs it. It then compares its results with the vendor’s reported figures, flags every discrepancy, and can leave the harness in place as a recurring gate for later rollouts. For model-level comparisons, the work runs as the LLM Selection Pack.
This engagement does not suit a pilot TechnoLynx itself delivered, because that would recreate the conflict the audit exists to remove. It covers the engineering evaluation. Clinical, legal or regulatory sign-off stays with the qualified function in your organisation. Other routes are realistic too. An internal second-line or compliance team with no stake in the selection can do it. So can a provider with the technical and managerial independence IEEE 1012 defines.
The assumption no package can settle in advance
Every section above eventually comes back to one number nobody can measure before purchase: average utilisation over the life of the hardware. The evidence package does not remove that uncertainty. What it does is make the uncertainty visible, bound it with a range, and tie it to a decision rule. The most useful thing to write into the case is when it gets re-run. Run it again when the pilot’s measured utilisation arrives, when a phase gate reports real throughput and power, and at each budget cycle, using scores from the same major release. A case built that way can say what would change its answer.
