Yes, a shared job queue can pin a job to one GPU SKU when the scheduler underneath exposes GPU model labels or types. Kubernetes node selectors, Kueue ResourceFlavors and Slurm GPU types restrict placement to a named model, and Ray’s accelerator types require a node of that type. Logging which SKU ran each job is a separate job that needs its own wiring.

TechnoLynx’s Inference Cost-Cut Pack helps you decide whether ring-fencing a single-SKU pool for an accuracy-sensitive customer tier is worth its cost compared with keeping that tier in the shared mixed pool.

Can the provider pin a job to one GPU SKU, or only log which one it ran on?

If you run your own Kubernetes or Slurm, pinning is in your hands, because the GPU model is visible as a node label or GRES type. On GKE Standard, GKE attaches a cloud.google.com/gke-accelerator node selector to GPU nodes, and a Pod that sets it lands only on that accelerator type.

Above that layer, things get murky. We could not find any public documentation of what provider-managed shared queues reveal about node SKU, so treat this as something to ask your provider, not something to assume either way.

Which scheduler features constrain a job to a single GPU SKU?

Kubernetes, Kueue, Volcano, Slurm and Ray all offer a way to pin a job to a GPU model. What differs is whether the constraint is required or only a preference.

In Kubernetes, the recommended approaches all use label selectors. nodeSelector is the simplest: the Pod is scheduled only onto nodes carrying every listed label. Node affinity comes in two kinds. requiredDuringSchedulingIgnoredDuringExecution means the scheduler cannot place the Pod unless the rule is met, while preferredDuringSchedulingIgnoredDuringExecution means it tries to, and still schedules the Pod somewhere else if no matching node is free. Preferred rules add a weight between 1 and 100 to a node’s score. In practice, required affinity pins and preferred affinity only nudges.

You match on nvidia.com/gpu.product, with values such as NVIDIA-A100-SXM4-40GB or Tesla-T4. NVIDIA’s GPU Feature Discovery, deployed with Node Feature Discovery, provides these labels, and NFD can also add taints. Taint the GPU nodes and give tolerations only to the workloads meant for them, so render jobs and GenAI jobs cannot drift onto each other’s hardware by accident.

The schedulers side by side

Scheduler	How a job is pinned to a SKU	Hard or soft	Watch for
Kubernetes	nodeSelector or required node affinity on nvidia.com/gpu.product	Hard (preferred affinity is soft)	Preferred rules still place the Pod elsewhere when no match is free
GKE Standard	cloud.google.com/gke-accelerator node selector, one node pool per accelerator type	Hard	Using several GPU types requires several node pools
Kueue	ResourceFlavor with node labels, taints and tolerations; labels injected into the Pod nodeSelector	Hard	Two GPU models behind one nvidia.com/gpu resource are not told apart without flavors
Volcano	Nodegroup labels with node group affinity set on the queue	Required rules hard, preferred rules soft	The strict flag decides whether queues without affinity may use unlabelled nodes
Slurm	--gpus=type:count, --gres=gpu:type:count, or --constraint on node features	Hard	GPU types come from the Gres configuration; features are defined by the cluster administrator
Ray	accelerator_type, or label_selector on node labels	Required (label_selector is experimental)	fallback_strategy adds soft constraints

A ResourceFlavor ties a class of hardware, such as A100 versus T4, to nodes through labels, taints and tolerations. When Kueue admits a workload, it adds the flavor’s labels to the Pod template’s nodeSelector (unless they are already there), which guarantees the Pods can run only on nodes targeted by the assigned flavor. Without that step, Kueue’s own documentation warns that two GPU types exposed as the same nvidia.com/gpu resource may not be differentiated, so Pods can land on either. Creating one flavor per hardware type, each with its own taint, means only workloads that tolerate that taint can use its quota.

The Kempner Institute’s cluster, for example, tags partitions with features such as a100, h100 and h200, which jobs select through --constraint.

One caveat applies to labels on shared GPUs. NVIDIA’s GPU Feature Discovery notes that the nvidia.com/gpu.product value may be modified by the device plugin when a sharing strategy is configured, and under the MIG “single” strategy the GPU labels describe MIG devices instead of full GPUs. If you partition with MIG or time-slice, check the actual label values in your cluster before writing affinity rules against them.

How do you record which GPU SKU each job or request actually ran on?

Recording the SKU means joining where the job or request was dispatched with what hardware sits there.

On the hardware side, NVIDIA’s DCGM Exporter always labels GPU metrics with gpu, UUID, pci_bus_id, device and modelName. With the right options enabled it also adds the Pod UID and Pod labels, and an hpc_job label when its job-mapping directory maps a GPU or MIG instance to a scheduler job. Ray’s system metrics carry GpuDeviceName, GpuIndex and GpuUuid on per-GPU series, alongside a JobId label. On Slurm, the GRES plugin sets CUDA_VISIBLE_DEVICES for each job step to the GPUs that step may use, which you can write into the job’s log at start.

vLLM exposes request-level histograms (time to first token, end-to-end latency and others) and labels every metric with model_name. Triton reports request counts per model and GPU utilisation, power and memory per GPU, with GPU metrics collected through DCGM. Neither ties an individual request to a device identity in the metrics listed. For per-request SKU records, we would stamp the node name and GPU UUID into the request log at dispatch time, then join against DCGM’s modelName.

Log the UUID, not just the model name. The PAL study (2024) points out that GPU clusters show significant performance variability both within and across machines, and that typical placement policies wrongly assume GPUs of the same architecture perform equally. Pinning the SKU does not pin performance; when a result looks off, the device identity is what lets you trace it to one card. Slurm’s documentation adds a further reason to record identity carefully: the mapping between device files and NVML’s PCI-ordered numbering is nondeterministic and can change between boots.

Remember what a log can and cannot do. Logging without pinning explains a result after the fact; it does nothing to make the next run land on the same hardware.

Can outputs differ between GPU models, and why does that matter for accuracy-sensitive work?

They can, even with the same model, prompt and software stack. A 2026 arXiv study on LLM inference across A100, L40S and H100 traces the cause to floating-point non-associativity combined with hardware-dependent kernel selection: each architecture picks different matrix-multiplication kernels with different reduction orders, and the rounding differences can flip output tokens. Run-to-run determinism on one device is typically guaranteed, but nothing forces two architectures to choose the same kernel. In that study, unmitigated BF16 diverged on 30.81 to 100% of problems across GPUs.

The effect shows up in evaluation scores, not just bits. A 2025 study found DeepSeek-R1-Distill-Qwen-7B varying by up to 9% in accuracy and 9,000 tokens in response length as GPU count, GPU type and evaluation batch size changed. The same authors note A100s showing slightly higher token-probability variance than L40S under identical configurations, with all such effects most pronounced in BF16.

That is the case for SKU pinning on accuracy-sensitive tiers: if a customer validated outputs on one GPU model, serving them on another can change what they see. Pinning the SKU is not the whole fix, though, because batch size, GPU count and tensor parallelism also change reduction order. The other route is to make kernels reproducible across hardware. The 2026 study’s fixed-configuration kernels made linear-layer outputs bitwise identical across all three architectures, at a median cost of 1.4% and a worst case of 21.9% over the served shapes, though attention, normalisation and sampling kernels stayed architecture-dependent. For a team that cannot pin hardware, that route is worth weighing.

When is a dedicated single-SKU pool the only hard guarantee?

A labelled shared pool with a required constraint places a job only on matching nodes. We would choose a dedicated pool in three situations:

• The provider’s abstraction hides the SKU, so there is no label to pin against.
• A contract promises the hardware itself, and you cannot risk a misconfigured job, a preferred rule or a fallback placing work elsewhere (Volcano, for instance, lets a queue’s tasks run on otherwise excluded node groups when preferred groups lack resources).
• The customer tier must not compete with other work for the pinned SKU at all, so capacity has to be reserved, not just constrained.

Picky GPU requirements carry a scheduling cost. The Alibaba MLaaS trace (NSDI 2022) of a 6,000-GPU production cluster describes hard-to-schedule tasks demanding high-end GPUs with picky requirements, long queueing delays and load imbalance across heterogeneous machines. Expect queueing delays on pinned jobs to rise during demand spikes, since they cannot spill onto free capacity of another model.

Whether to run one pool across vendors and SKUs or partition by hardware at all is a wider architecture question than the per-customer pinning covered here; the guidance on GPU performance engineering covers that work.

What a dedicated single-SKU pool costs compared with a shared mixed pool

The cost of a single-SKU pool shows up as under-used capacity. Two pieces of evidence put numbers on it, each measuring a slightly different thing.

The Mélange study measured LLM serving across L4, A10G, A100 and H100. Compared with using a single GPU type, a mixed allocation cut cost by 9 to 77% in short-context workloads, 2 to 33% in long-context ones and 4 to 51% in mixed-context ones. The reason is granularity: scaling with A100s alone is coarse and leaves capacity under-used, while mixing in cheaper GPUs matches capacity to demand more closely. Selecting one GPU type across all request sizes gave up as much as 72% more output tokens for the same cost. Those figures compare whole serving fleets, not one customer tier carved out of a pool, so read them as the size of the efficiency at stake, not as your premium.

Uptime’s analysis of dedicated clusters shows low utilisation raising unit cost sharply. It puts the breakeven against public cloud at 33% utilisation. At 8% utilisation an hour on dedicated equipment cost $250 against $66 for cloud, while at 40% it cost $50, a 25% saving. These figures compare owned clusters with public cloud, not a reserved pool with a shared one; they show that idle capacity raises unit cost.

Worked example: pricing a pinned tier

Our assumption, for illustration only: Uptime’s owned-versus-cloud unit costs stand in for a reserved pool at the utilisation shown.

1. Under that assumption, a pool kept busy 40% of the time costs about $50 per hour.
2. Under that assumption, a pool used 8% of the time costs about $250 per hour.
3. $250 against $50 is five times the unit cost at 8% versus 40% utilisation in Uptime’s comparison.

On this illustration, the guarantee is cheap for steady tiers and expensive for quiet ones.

A decision procedure for SKU guarantees in a mixed queue

1. Check what your scheduler exposes. If GPU model labels or types are visible, you can pin with required constraints; if they are not, ask the provider for the feature or move the tier to capacity you control.
2. Add SKU-specific queues or flavors (Kueue ResourceFlavors, Volcano node groups, Slurm types or features) for the jobs that need them, using required rules, not preferred ones.
3. Wire logging at dispatch: node and GPU UUID into the job or request log, joined against DCGM’s modelName.
4. Decide whether the guarantee must survive load spikes and misconfiguration. If not, a labelled shared pool with logging is enough.
5. Choose a dedicated single-SKU pool only when accuracy sensitivity or a contract outweighs the utilisation cost, and price it against the tier’s expected utilisation.

Who can help us decide whether a single-SKU pool is worth it?

TechnoLynx helps teams running mixed render and GenAI queues decide whether an accuracy-sensitive customer tier needs a pinned or dedicated pool, or whether constrained placement and logging in the shared pool will do.

It is not the right route for everyone. If your tier runs on a serverless GPU product that exposes no SKU, the decision is about your provider contract, and the provider’s account team is the first call. A platform team already fluent in Kubernetes or Slurm scheduling can add labels, flavors and dispatch logging itself; outside help matters most when the cost question is genuinely close.

FAQ

How do I check GPU workload on a shared cluster?

Scrape per-GPU metrics: DCGM Exporter publishes them with model, UUID and PCI bus labels, and Triton exposes GPU utilisation, power and memory per GPU on its metrics endpoint. On Ray, ray_node_gpus_utilization reports utilisation per GPU with the device name attached.