To shortlist hardware and a software stack for an AI compute workload, start with a documented rubric of weighted criteria you can actually check. Score each candidate for lock-in, compare the full delivered cost and risk, and decide in advance which events will trigger a re-test. None of those steps measures performance. LynxBenchAI treats empirical execution of real software workloads as the reference standard for accelerator performance. Estimating performance from specifications is an approximation, to be used only while running the workload is not yet possible. The shortlist’s job is to decide what is worth running. It does not replace the run.
Independent research reaches a similar conclusion. One study benchmarked computer-vision and large-language-model workloads on NVIDIA H100, NVIDIA H200 and AMD MI300X GPUs. It concluded that procurement decisions should rest on workload-specific throughput, energy efficiency and software-stack maturity rather than on peak specifications alone. Google Cloud’s vendor-neutral accelerator benchmarking guide puts it more bluntly: having the highest hardware specifications does not mean applications can make use of them.
Can we shortlist accelerators or SoC boards from spec sheets and briefing decks alone?
Yes, as long as the rubric weights what documentation can genuinely verify and treats vendor performance claims as unverified until something has been run. That means availability, SDK and framework maturity, memory fit, power envelope and whether the device has been independently measured. A documentation-only shortlist is a filter, not a verdict.
LynxBenchAI can close part of that gap before you get hands-on time. If a candidate device already appears in the AI Hardware Directory of LynxBenchAI results, its published scores give a measured input instead of a spec-sheet guess. The first step is to check which of your candidates the directory covers.
Performance claims belong at the bottom of the weighting because vendor figures may not be comparable across platforms. A benchmarking study of Intel, NVIDIA, AMD and Google accelerators notes that vendors provide their own benchmarks using their own highly optimised libraries and configurations, and that these results can be compared unfairly. Tim Dettmers’ GPU guide documents NVIDIA comparing GPUs at different batch sizes and GPU counts in ways that favoured the newer part. A TOPS figure, as Embedded.com explains, is the number of operations an accelerator can handle per second at 100% utilisation.
The UK government’s multi-criteria analysis manual describes using the technique to short-list a limited number of options for detailed appraisal. It says each criterion must be measurable, at least qualitatively, and that six to twenty criteria is the more typical range. A vendor-selection framework from TekRecruiter adds two rules worth adopting outright: use risk-based gates that disqualify a candidate automatically, and record the evidence beside each score, because “a score without evidence is just a stronger-looking opinion”.
A reusable rubric for documentation-only screening
| Criterion | Role | Evidence available before hands-on testing | Weight |
|---|---|---|---|
| Supported path for your framework and model (PyTorch build, runtime, operator coverage) | Gate | Vendor SDK documentation, framework support matrix, release notes | Pass or fail |
| Memory capacity against your estimated footprint | Gate | Spec sheet plus your own footprint estimate | Pass or fail |
| Purchasable now, with a published lifecycle date | Gate | Distributor stock, vendor lifecycle pages | Pass or fail |
| SDK and toolchain maturity | Scored | Release history, documentation depth, public issue tracker | High |
| Independent measurement coverage | Scored | Published LynxBenchAI or MLPerf results for the device | High |
| Lock-in score (see below) | Scored | Toolchain, format, second-source and exit evidence | Medium to high |
| Power and thermal envelope | Scored (a gate for edge and fanless designs) | Datasheet TDP and cooling requirements | Medium |
| Reader interest and coverage gaps (editorial reviews only) | Scored | Audience data, prior coverage | Medium |
| Vendor peak TOPS or FLOPS | Recorded, not scored | Briefing deck | None until measured |
You can apply the same rubric every cycle, with three checks from the same manual. First, drop any option that is dominated, meaning another performs at least as well on every criterion and strictly better on at least one. Second, confirm that each criterion can be scored without knowing the others, so that one effect is not counted twice. Third, test how sensitive the rankings are to changes in the criteria weights. Reviewers who pick boards for an editorial comparison piece are running a selection process, not a lab evaluation. Saying so in print, and publishing the weights, is what makes the selection defensible. Weights should change as the market and reader interest shift.
Letting vendor marketing material set the weights biases the shortlist.
Is there a best GPU for AI training, or only a best GPU for your workload?
There is only a best GPU for your workload. The clearest evidence is how badly peak figures predict measured throughput. In the study of Intel, NVIDIA, AMD and Google accelerators cited above, Google’s TPU v3-8 has about 2.3 times the peak FLOPS of the TPU v2-8, yet it measured only 1.5 to 1.7 times faster across three DNNs. The authors say FLOPS utilisation on the v3-8 is partially bottlenecked by its 1.5-times improvement in memory bandwidth. The same study found that a Tesla V100 more than doubled its own FP32 training throughput with mixed precision while using only 9% of its Tensor Core peak FLOPS.
The LynxBenchAI 26Q3 results show the same thing between two consumer cards. The NVIDIA GeForce RTX 4090 scores 367.9 against the RTX 3090’s 87.4 in the compute category, which is built from matrix-multiplication tests. In training the scores are much closer: 335.9 against 207.6. A raw-compute comparison would overstate the newer card’s lead for a training job.
The architecture of the model matters as well as the silicon. Dettmers observed that large-transformer performance across GPU architectures tracks memory bandwidth closely. CNN speedups vary much more: networks built on grouped or depth-wise separable convolutions gained only 1.2 to 1.5 times, while ResNet-style networks gained 1.6 to 1.7 times. Google Cloud’s guide adds that a model co-designed for one platform may not fully use another, or even a later generation of the same platform.
To check a training-GPU shortlist beyond the spec sheet, run a representative model on each candidate and normalise the results. Google’s guide normalises to throughput per chip with the model architecture held constant, then divides by chip cost. Turning those runs into a decision that survives finance review, with validation and total cost of ownership, is covered in justifying compute stack choices with benchmarks, validation and total cost.
Which GPU and stack should we use for 3D CNN training on whole-slide pathology images?
PyTorch with MONAI is the natural stack to start from; A100-, H100- and A6000-class GPUs are candidates to test in the pilot, not choices backed by a published whole-slide comparison. However, the choice for 3D CNN training on gigapixel whole-slide images has to be settled by a pilot run on your own slides. MONAI is an open-source, consortium-led framework that extends PyTorch for medical data, with imaging-specific architectures, transforms and utilities.
LynxBenchAI measures how covered GPUs run the public PyTorch stack, so the first step is to read its published scores for your candidate GPUs before planning the pilot.
LynxBenchAI’s published results show how a covered GPU runs the public PyTorch stack, which makes them broadly indicative for custom PyTorch models. They are not a whole-slide 3D CNN benchmark. The practical first step is to read the scores for any covered candidates and then run LynxBenchAI on the pilot node.
Memory is a key constraint for these inputs. According to Dettmers, input size is the biggest driver of CNN memory use, very large medical images need a great deal of it, and for branched architectures you need to benchmark the actual network to find out where the memory goes. So a memory estimate made from the architecture alone needs confirming by running the actual network.
No single figure answers the minimum viable cluster size. Size the smallest configuration that meets the memory and throughput needs of a representative slide set, pilot it, and scale only from measured epoch times. Two things carry over from general GPU guidance. Dettmers says in a reply that the A100 and A6000 come close on cost-performance, and a cluster mixing GPU types runs data-parallel training at the speed of the slowest card.
In a validated research environment, freeze the driver, CUDA, PyTorch and MONAI versions for the pilot and record them, so that the pilot result describes the environment you will validate. Benchmark results for other imaging tasks may not transfer to gigapixel 3D CNNs, and budget-driven sizing estimates still need confirming on your data and model.
Is a used dual RTX 3090 rig a better buy than a single RTX 4090 for fine-tuning?
It is only a better buy if your fine-tuning job needs more than 24 GB of GPU memory and your framework can split the job across two cards. If the job fits in 24 GB, the single RTX 4090 scores higher as a single card in 26Q3.
LynxBenchAI publishes single-card scores for both cards on the public PyTorch stack, so the first step is to compare their published results before pricing a build.
LynxBenchAI’s 26Q3 results score the RTX 4090 above the RTX 3090 in every category: 335.9 against 207.6 in training, 659.4 against 294.1 in inference and 367.9 against 87.4 in compute. These are single-card scores on the public PyTorch stack. They are broadly indicative for custom PyTorch models, but the release does not measure a two-card 3090 pair or your fine-tuning job. Precision support also differs. The RTX 3090 produced no FP8 inference result in 26Q3, while the RTX 4090 scored 673.1 at FP8 on DistilBERT inference.
A separate vendor benchmark points the same way. It measured RTX 4090 TF32 training throughput at 1.3 to 1.9 times that of the RTX 3090, depending on the model, and training throughput per dollar at 1.2 to 1.6 times.
Counting the full build, not just the cards
| Memory | 24 GB per card | 24 GB |
| Card power | 350 W each, so 700 W for the pair before CPU and board (2 × 350 W) | 450 W; NVIDIA recommends at least 850 W system power |
| Slots and board | Three-slot cards; both slots should run at least x8/x8, because a second slot wired at x4 costs about 15% throughput in one build guide | Needs 3.5 PCIe slots of width |
| NVLink | The RTX 3090 is the last GeForce card with NVLink | The RTX 40 series has no NVLink connector |
| Usage history | Unknown before purchase | New |
Used prices move, so treat any such figure as a snapshot. Dettmers’ guide puts a more powerful power supply at about $50 per GPU.
NVLink is not required for a dual-3090 rig running llama.cpp. It pays off when a workload is split across GPUs that exchange data constantly, whereas llama.cpp’s default layer split takes turns between cards and needs only PCIe. Two 3090s also do not act as one 48 GB GPU automatically. Splitting a model across them depends on model parallelism, which Dettmers notes is less widely supported than data parallelism.
The used-market risk is real but hard to price, because silent wear is not visible before purchase. Before committing to a pair, run a sustained stress test on each card on arrival and compare it with the 26Q3 single-card scores using LynxBenchAI’s Personal Edition.
How do we compare cloud GPU rental price-performance across providers?
Lambda’s RTX 3090 instances, like any provider’s, should be compared on performance per dollar for your workload rather than on the hourly rate. List prices for the same GPU vary widely: AltStreet’s July 2026 verification pass found H100 pricing from about $2.46 to $12.29 an hour, a fivefold spread. TechnoLynx can scale down your own PyTorch training or inference job, run it across candidate providers and GPU SKUs, log availability and price movement over the test window, and package the results as a citable report for a grant justification.
Who can measure our shortlist on real hardware and give us results we can cite?
LynxBenchAI gives you measured results for the devices it covers. Each release scores devices running the public PyTorch stack that teams’ own models run on, so the scores are broadly indicative for custom PyTorch models. The Personal Edition runs the same suite on hardware you have access to. For your own workload, TechnoLynx runs a scaled-down version across candidate GPUs and providers and reports throughput and cost per unit of work. You can start from LynxBenchAI.
There are limits. A published score shows how a device runs the public PyTorch stack in the LynxBenchAI suite. It does not run your model, and it does not tell you whether your pipeline meets its SLA. There are other fair routes. MLPerf Inference publishes reproducible measurements made under common rules by many organisations. NVIDIA’s vGPU sizing guide recommends a proof of concept on the target GPU to establish baseline framebuffer requirements. A harness you build yourself is another option, provided you pin versions and warm-up rules.
How do we score vendor lock-in risk objectively across candidate platforms?
Score lock-in as a weighted composite of four dimensions: toolchain openness, model-format portability, second-source availability of compatible hardware, and exit cost. Apply the same documented rubric to every platform, and treat an open format as a gate rather than the whole score.
LynxBenchAI scores executors, a device plus its software stack. Running the Personal Edition on a second candidate’s stack shows whether the portable path performs, not only whether it exists.
The scoring approach has precedent. A paper on AI procurement in the ITEA Journal recommends migration cost estimates, dependency mapping and risk-scoring frameworks that weight each dependency by how hard it is to mitigate and how much operational damage it could do. ITAM Review’s 2026 guide reduces the exit side to three questions: how long would it take to leave, what would it cost, and what would you lose. It adds a useful test: if you cannot describe your exit on a single page, you do not have enough information to negotiate.
| Dimension | What to score | Evidence |
|---|---|---|
| Toolchain openness | Whether compiler, runtime and kernels are open, documented or vendor-only | SDK licences, source availability, supported targets |
| Format portability | Whether the model exports cleanly to ONNX without framework-specific operations | A real export and a test of the converted model |
| Second source | Whether another vendor’s hardware runs the same path | A measured run on the second platform |
| Exit cost | Exit time, cost and capability lost | A one-page exit plan |
Should a mandate require open compiler toolchains, or only open model formats? Require open formats as the baseline gate, because ITEA’s authors argue that planning ONNX export from the start of a project is the most realistic defence against framework lock-in. Then score toolchain openness on top, because a portable format does not make performance portable. Model lock-in, as the same paper notes, arises when performance depends on vendor-specific optimisation libraries. An industrial machine-vision benchmark shows the size of the gap: on a Jetson CPU, OpenVINO took 102,720 ms for Grounding DINO against 13,580 ms for ONNX Runtime, more than seven times slower.
Public bodies have set precedents. The UK government’s open standards principles require bodies to publish the open standards they use, their exemptions and their exit management strategies, and to back an exemption from a compulsory standard with an impact analysis. In November 2025 the US Department of War required contractors to maintain at least two qualified sources for critical programme content, specifically to prevent lock-in. Precedents may not transfer if the hardware category or regulatory context differs. The rubric’s weights are value judgements, and the multi-criteria manual is explicit that scoring and weighting embed the decision-makers’ preferences. Publish the weights so stakeholders can challenge them.
How often should we re-validate the shortlist, and what should trigger a re-test?
Let specific events trigger re-tests (a driver branch change, a new SDK or framework major version, a firmware update, an end-of-life or change notice). Keep a calendar interval only as a backstop for modules where nothing has fired.
LynxBenchAI gives you a fixed suite to rerun after each trigger.
Aviation certification offers a useful default. DO-330’s requalification taxonomy, documented in Pothon et al., treats a changed tool as requiring requalification, scoped by an impact analysis. Where only the environment changes, the default is invalidation unless equivalence is demonstrated. In other words, a shortlist entry should be assumed stale after a stack change until a re-test shows otherwise. NVIDIA’s vGPU sizing guide makes the same point in its own domain: do not assume that a profile validated on one GPU generation or software stack still holds after either changes.
| Trigger | Why it matters | Cadence evidence |
|---|---|---|
| New production driver branch, or moving off one | NVIDIA releases two data-centre production branches a year, each supported for one year; long-term support branches get three years | Roughly every six months |
| New major version of CUDA, PyTorch, TensorRT or the vendor SDK | A changed software stack invalidates prior sizing assumptions | Per vendor release |
| Firmware update | Changes the tested stack, so prior results are assumed stale | Per vendor release |
| End-of-life notice | NVIDIA sends Jetson end-of-life notices at least 8 months before last shipment | Per notice |
| Product change notice | Jetson hardware component changes are posted as PCNs under JEDEC JESD-046 | Per notice |
| Calendar backstop | Catches silent drift where no notice was published | Align it with the driver branch cadence |
Keep a living registry with one row per module and version. Record its status, the date of its last test, the trigger that caused that test, and a sunset or supersede rule. Give the registry a named owner, because without one it decays. Three caveats apply. The calendar backstop covers changes for which no notice is published. Minor patch releases should not trigger a full re-test, so that testing cost stays proportionate to the risk. And every re-test should follow the same protocol; a reproducibility checklist for fair, trustworthy benchmarks is the right companion to the registry.
If stock PyTorch misses our latency SLA, what is the cheapest fix?
The cheapest fix is the one that closes the SLA gap at the lowest combined engineering and infrastructure cost, and measurement on your model shows which it is. A different GPU, TensorRT or ONNX Runtime conversion, and a change of batching strategy have no fixed order of merit.
LynxBenchAI provides a stock-PyTorch baseline per device. Its published results show how each device runs the public PyTorch stack, which tells you whether a hardware change alone is plausible before anyone converts a model. When conversion or batching is the lever, TechnoLynx’s Inference Cost-Cut Pack cuts inference cost and p95 latency on the workload you actually run, starting with an audit.
How much latency conversion buys depends on the model. One study of BERT-base on an RTX 3090 reported TensorRT FP16 at 2.1 ms for batch size 1. Breaking that down, GPU execution contributed 5.4 times, ONNX Runtime a further 1.2 times, TensorRT graph optimisation 1.5 times and FP16 precision 1.9 times. The machine-vision benchmark found a different pattern on transformers. TensorRT cut YOLOv8 latency by roughly 30% to 56% against the next-best framework on NVIDIA GPUs, but for the transformer-based Grounding DINO, plain PyTorch tended to be fastest on the desktop GPUs.
Conversion has costs that a latency figure does not show. In the BERT study, blanket FP16 lowered SST-2 accuracy by 0.34 percentage points, while a hybrid scheme that kept softmax and layer normalisation in FP32 preserved it. TensorRT engines also took 142 to 213 seconds to build, which adds to deployment time. Batching changes trade latency against throughput depending on traffic shape. If one lever does not close the gap, combine levers, but only after measuring each one on its own.
What a finished shortlist should hand to the people who run it
A finished shortlist should contain the rubric scores with evidence beside each one, a lock-in score with its exit page, the cost basis per unit of work, the list of re-test triggers, and the specific runs still to be done. Mark every number in it as either documented or measured. The documented ones are the shortlist’s open debts, and the measurement step exists to pay them off.
FAQ
Is an accelerator the same as a GPU?
No. “Accelerator” is the umbrella term, and a GPU is one category of accelerator alongside TPUs, NPUs, XPUs and CPUs used for AI compute.
What is the NVLink speed of the RTX 3090?
The RTX 3090 uses NVLink 3 at 112.5 GB/s, compared with 600 GB/s for the A100 on the same NVLink generation.
How cheap are cloud GPUs?
In AltStreet’s July 2026 verification pass, RTX 4090 instances started at about $0.14 an hour on spot pricing from decentralised marketplaces, and A100 80GB instances ran from about $1.21 to $2.79 an hour at specialised providers.
