“We installed OpenCL months ago, it works.” That sentence is usually true on the box it was installed on and quietly false the moment you move the serving workload to a different accelerator. OpenCL installation looks like a one-time driver step — apt-get something, reboot, done — but on a production inference path it decides whether your serving stack stays portable across hardware or gets pinned to one vendor’s economics. The naive install is easy to spot in a Dockerfile: install whatever GPU vendor package the base image ships, never touch the ICD loader, never pin the runtime version. It runs. It also strands you. The expert install treats the OpenCL runtime, the ICD registry, and device selection as reproducible parts of the serving image — so that a hardware swap or a model swap does not silently change your cost-per-request. That difference is invisible until you try to move the workload, and by then the copy-pasted install has already made the decision for you. How does OpenCL installation actually work? OpenCL is not a single thing you install. It is a loader plus one or more implementations, and confusing the two is where most production setups go wrong. The piece your application links against is the ICD loader — the Installable Client Driver loader, usually shipped as libOpenCL.so (the ocl-icd package on most Linux distributions, or Khronos’s reference loader). The loader itself computes nothing. When your process calls clGetPlatformIDs, the loader reads a registry of .icd files under /etc/OpenCL/vendors/, each of which is a one-line pointer to a vendor’s actual runtime library. The loader dispatches your calls to whichever runtime that registry names. So an OpenCL install is really three separate concerns stacked on top of each other: The loader — one per system, vendor-neutral, links your app. The runtimes — one per accelerator vendor (NVIDIA’s OpenCL runtime, AMD’s ROCm OpenCL, Intel’s Compute Runtime, or a CPU fallback like PoCL). The ICD registry — the /etc/OpenCL/vendors/*.icd files that tell the loader which runtimes exist. The reason this matters for inference: your serving code links the loader, but the loader picks the runtime at process start based on that registry. If the registry silently changes between environments — because the base image shipped a different vendor package, or because a driver update rewrote an .icd file — the same binary runs on a different device with different per-request cost, and nothing in your application logs will tell you why. This is the mechanism behind portable GPU inference backends on Linux: the portability lives in the loader-plus-registry seam, not in the runtime. What the ICD loader and runtime mean for a portable serving path The whole point of OpenCL for a production serving stack is that one binary can target multiple accelerator families without recompilation. That promise is real, but it is delivered entirely by the loader/runtime split — and it only holds if you install both halves deliberately. Here is the failure mode we see most often in practice: a team builds a serving image on a workstation with an NVIDIA card, the vendor package drops an nvidia.icd into the registry, everything works, and the image ships. Later, the platform team wants to evaluate the same workload on AMD or Intel silicon for cost reasons. They deploy the image, it falls back to a CPU runtime (or fails to find a device at all), throughput collapses, and the conclusion becomes “OpenCL doesn’t work on that hardware.” It does. The image just never installed the AMD or Intel runtime, and its registry only ever pointed at NVIDIA. A correct install decouples these. The loader is constant; the runtimes present in the image are an explicit, versioned choice; and device selection at startup is checked, not assumed. When that holds, cost-per-request becomes measurable across targets — the exact precondition for treating hardware as a variable you can optimize rather than a constant you’re stuck with. When it doesn’t hold, you have a single-vendor serving path wearing a portability costume. How do you verify an OpenCL install picks the right device? Do not trust that the install worked because the process started. A CPU-fallback runtime will start cleanly and serve requests — slowly and at a completely different cost profile. Verification means confirming, at image-build time and at container start, that the loader sees the runtime you intended and that device selection lands on the accelerator you’re paying for. Diagnostic checklist: is this OpenCL install serving from the device you think it is? Run these in order. Any surprise means the install is not what your Dockerfile implies. clinfo reports the expected platform and device. The Platform Name and Device Name fields should name your target accelerator, not Portable Computing Language (PoCL) or a CPU device — unless CPU is what you meant. /etc/OpenCL/vendors/ contains exactly the .icd files you installed. An extra .icd from a base-image package is a silent second device that device-selection order may pick over yours. The runtime version is pinned, not floating. Note the reported driver/runtime version and pin the package that supplies it. A floating version means a future rebuild can change the numerical behaviour of your kernels. Your serving process’s device index resolves to the accelerator, not device 0 by default. Enumerate devices explicitly rather than assuming index 0; on multi-device hosts index 0 is not guaranteed to be your accelerator. A trivial kernel actually runs on-device. A CPU-fallback runtime passes checks 1–4 in isolation but a timed micro-kernel exposes it — device throughput and CPU throughput differ by orders of magnitude. The self-contained test above is the difference between “the container started” and “the container is serving from the device whose price you budgeted.” We treat step 5 as non-negotiable in a serving image: it is the only check that distinguishes a working accelerator path from a plausible-looking CPU fallback. How should OpenCL setup live in a reproducible serving image? Ad-hoc installs are the root cause of every “works on the build box, wrong device in prod” incident. The fix is to move the loader, the runtimes, and the registry into the image definition as explicit, pinned steps — the same discipline you’d apply to any dependency that affects numerical output. The comparison below is the practical divergence point named in the setup: two Dockerfiles that both “install OpenCL” but produce very different portability and cost properties. Concern Ad-hoc install Reproducible install Loader Whatever the base image happened to ship ocl-icd (or Khronos loader) installed and pinned explicitly Runtimes One vendor’s, inherited implicitly Each target vendor’s runtime installed as a named, versioned layer ICD registry Unaudited /etc/OpenCL/vendors/ Registry contents asserted at build time; unexpected .icd files removed Device selection Defaults to device 0 Enumerated and validated at container start Version drift Floating; a rebuild can change kernel numerics Runtime version pinned; rebuilds are deterministic Hardware swap Requires a new build + debugging session Same image, validated against a new target in one profiling pass The payoff of the right column is concrete and measurable: device-utilisation at the target accelerator, the cost-per-request delta after moving between OpenCL-capable devices, and a sharp reduction in image-rebuild time when you validate a new hardware target (observed pattern across our inference-cost engagements, not a benchmarked rate). When the install is reproducible, evaluating a new accelerator is a profiling task, not an archaeology project. This is the same reproducibility logic that governs how machine learning compilers cut cost-per-request — the runtime choice and its version are inputs to the numbers, so they have to be pinned like any other input. How a correct install affects cost-per-request across hardware Cost-per-request is set by three things: how much device time a request consumes, how fully the device is utilised, and what that device costs per hour. OpenCL touches all three, because the runtime you install determines how your kernels actually execute on the silicon. A reproducible install protects the cost-per-request SLO by keeping the serving path portable rather than pinned to one accelerator’s pricing. That is the whole ROI argument. If your image can only run on one vendor’s runtime, you are locked into that vendor’s cost curve — and when their spot pricing moves or a cheaper accelerator becomes available, you can’t act on it without a rebuild-and-debug cycle that may take longer than the pricing window stays open. A portable install turns “can we run this cheaper elsewhere?” into a one-pass measurement instead of a project. Profiling that per-request cost across the OpenCL device path is itself a discipline — a correct install is the precondition for it, not a substitute. The GPU profiling methodology that underwrites per-request cost assumes the device path is stable and known; an unaudited install invalidates that assumption before you take a single measurement. If your serving stack runs at AI-infrastructure scale, the AI infrastructure and SaaS practice is where this connects to unit economics: the install is where portability is won or lost, and portability is where cost optionality lives. What installation mistakes silently degrade utilisation or portability? The dangerous mistakes are the ones that don’t error. A build that fails loudly gets fixed. A build that succeeds on the wrong device ships and quietly inflates your bill. The recurring ones we see: Trusting the base image’s OpenCL. Many GPU base images ship a runtime and a registry entry you didn’t audit. You inherit a device selection you never chose. Never pinning the runtime version. A rebuild months later pulls a newer runtime, kernel numerics shift subtly, and an eval that used to pass now drifts — with no code change to blame. Leaving multiple .icd files in the registry. Device enumeration order is not guaranteed; a stray CPU or integrated-GPU .icd can win device 0 and halve utilisation. Skipping the on-device kernel check. Every check passes, the container serves, and only the cost report reveals it’s been running on a CPU fallback for weeks. Treating the loader and runtime as one package. Installing a vendor runtime without an explicit loader (or vice versa) produces a link that works by accident and breaks on the next base-image bump. None of these throw an exception. All of them change cost-per-request. That is precisely why the install belongs in version control alongside your model and your serving config, not in a run-once setup script someone executed on a workstation eighteen months ago. FAQ What matters most about OpenCL installation in practice? An OpenCL install is really three layered concerns: a vendor-neutral ICD loader (libOpenCL.so) that your application links against, one or more vendor runtimes that do the actual compute, and an ICD registry under /etc/OpenCL/vendors/ that tells the loader which runtimes exist. In practice, the loader picks a runtime at process start based on that registry — so “installing OpenCL” means deliberately choosing all three parts, not just apt-getting a driver. What are the ICD loader and runtime, and why do they matter for a portable serving path? The ICD loader is the constant, vendor-neutral piece your binary links against; the runtime is the per-vendor implementation that runs kernels on a specific accelerator. Portability — one binary targeting multiple accelerator families — is delivered entirely by keeping the loader constant while installing each target vendor’s runtime explicitly. If only one vendor’s runtime is present, the serving path is single-vendor no matter what the loader promises. How do I verify an OpenCL install picks the right accelerator device for inference? Don’t trust that the process started, because a CPU-fallback runtime starts cleanly and serves slowly at a different cost profile. Verify that clinfo names your target device, that /etc/OpenCL/vendors/ holds only the .icd files you installed, that the runtime version is pinned, that device selection resolves to your accelerator rather than defaulting to index 0, and — the decisive check — that a trivial timed kernel actually runs on-device. How should OpenCL setup be captured in a reproducible serving image rather than done ad hoc? Move the loader, each target runtime, and the registry into the image definition as explicit, pinned layers, and assert the registry contents and device selection at build and container start. Ad-hoc installs inherit whatever the base image shipped, float the runtime version, and default device selection — the root cause of “works on the build box, wrong device in prod.” A reproducible install makes rebuilds deterministic and turns a hardware swap into a single profiling pass. How does a correct OpenCL install affect cost-per-request when moving a workload across hardware? Cost-per-request depends on device time per request, device utilisation, and device hourly cost — and the installed runtime affects how kernels execute on all three. A reproducible, portable install protects the cost-per-request SLO by keeping the serving path movable, so “can we run this cheaper elsewhere?” becomes a one-pass measurement instead of a rebuild-and-debug project locked to one vendor’s pricing curve. What common installation mistakes silently degrade device utilisation or portability? The dangerous mistakes don’t error: trusting the base image’s unaudited OpenCL, never pinning the runtime version, leaving stray .icd files that win device 0, skipping the on-device kernel check, and treating the loader and runtime as one package. None throw an exception, all change cost-per-request, and that is why the install belongs in version control beside your model and serving config. Where the install stops being a driver step The tell that an OpenCL install has become an engineering artifact rather than a setup chore is when you can answer “what does this workload cost on a different accelerator?” without opening a debugging session. If the answer requires rebuilding the image, tracing which .icd file won device 0, and confirming a kernel isn’t silently on CPU, then the install is still deciding your economics for you — and it decided them a long time ago, on a workstation nobody remembers. A correct OpenCL install is the precondition our inference cost-cut work depends on to profile and compare per-request economics across a deployed serving path; the failure class it prevents is the single-vendor serving path that looks portable and isn’t.
