A RISC-V server host is driving your GPU. Utilisation reads 95%, so the team concludes the kernel is the bottleneck and starts re-tuning it. That conclusion may be wrong — and on an emerging host architecture, it is more likely to be wrong than on a mature x86 box. The reasoning that gets teams there is worth naming plainly: once a GPU is attached, the host CPU is treated as interchangeable plumbing. Profile the kernel, watch the utilisation counter, declare the workload compute-bound, and move on. That works often enough on well-understood x86 hosts that it hardens into a habit. RISC-V servers break the habit — not because the architecture is worse, but because the parts of the stack that make a host feel invisible are still maturing. How should you think about a RISC-V server in practice? RISC-V is an open instruction set architecture (ISA). A RISC-V server is a datacentre host built around a RISC-V application-class CPU instead of an x86 or Arm one — running Linux, addressing PCIe devices, and hosting the same class of accelerators you would attach to any other server. For years these existed mostly as evaluation boards. That is changing: application-class RISC-V silicon with vector extensions and multi-core layouts is showing up in real host roles, and the practical question shifts from “does it boot” to “does it drive an accelerator well.” The word “server” matters here. A server host does a specific job in a GPU workload: it copies data across PCIe into device memory, it issues kernel launches through a driver, it handles interrupts and completion signals, and it manages the memory the GPU reads from. None of that is GPU work. All of it is host work. The ISA of the host CPU shapes how fast and how predictably that host work happens — which is exactly the part the “host is irrelevant” assumption erases. This is the same framing we apply to any non-x86 or emerging host, and it sits inside the broader question of what the open ISA means for portable accelerated workloads. This article narrows to one slice of that: what a RISC-V host changes about profiling. How does a RISC-V host CPU differ from x86 when driving a GPU workload? The difference that bites is rarely raw compute. It is the maturity and behaviour of everything between your application and the device. Consider the path a single inference request takes. Input data lands in host memory. The runtime — PyTorch, ONNX Runtime, TensorRT — stages that data and issues a cudaMemcpyAsync or equivalent over PCIe. The driver validates the call, translates it, and pushes work onto the device queue. The kernel runs. Completion signals come back, the host reads results, and the loop repeats. On a mature x86 host with a hardened driver stack, each of those host-side steps is fast, predictable, and largely invisible on a profiler trace. On a RISC-V host, three things can change. First, the GPU vendor’s driver and userspace libraries may be less optimised for the platform, or built through a compatibility path rather than a native one, adding dispatch overhead per launch. Second, host-side single-thread performance and memory-subsystem behaviour differ, and dispatch-heavy workloads that launch thousands of small kernels are sensitive to exactly that. Third, PCIe transfer setup and the NUMA topology the host exposes may not be tuned the way x86 platforms have been for a decade. We treat these as host architecture questions, not GPU questions — the same way ACPI SRAT and cache-as-NUMA-domain effects shape data movement on any host. None of this means a RISC-V host is slow. It means the host is legible as a variable in a way it usually is not, and profiling has to account for it. Can a RISC-V host become the bottleneck even when GPU utilisation looks high? Yes — and this is the single most important thing to internalise before you profile a RISC-V server. GPU utilisation, as reported by tools like nvidia-smi, measures whether the device had any work scheduled on it during a sampling window. It does not measure whether that work was useful, saturating, or waiting. A workload can show 90%+ utilisation while the GPU spends much of each interval stalled on the next memcpy or blocked waiting for the host to issue the next launch. The counter goes up because the device is “busy” in the accounting sense; the wall-clock throughput stays low because the real constraint is upstream. This is the same trap we describe when separating metrics that explain GPU bottlenecks from metrics that mislead — high utilisation is a presence signal, not a saturation signal. On a slower or less-optimised host, the gap between “device had work scheduled” and “device was doing useful work at full rate” widens. A dispatch-bound loop that is comfortably hidden on x86 can surface as a genuine host-side bottleneck on RISC-V. The workload is host-bound. The utilisation counter says compute-bound. Believing the counter sends you re-tuning a kernel that was never the problem. Quick answer: is my RISC-V workload actually GPU-bound? Symptom Likely reading What to check next High utilisation, low throughput, many small kernel launches Host dispatch overhead Per-launch host time in a timeline trace Throughput scales poorly as batch size grows PCIe transfer / staging memcpy duration vs kernel duration GPU idle gaps between kernels on the timeline Host-side stall CPU thread activity during the gaps Utilisation and throughput both high and stable Genuinely compute-bound Now kernel tuning is justified The last row is the only one where re-tuning the kernel first is the right move. The other three point back at the host — and on a RISC-V server, the first three are more common than habit expects. (Thresholds here are illustrative diagnostic cues, not benchmarked values; the point is the direction to look, not a fixed number.) What does a profiler trace look like on a RISC-V host, and where does dispatch overhead show up? A timeline profiler — Nsight Systems, or an equivalent that captures both host and device activity — is the tool that settles the argument, because it puts host and device on the same clock. You are not looking at a single aggregate utilisation number. You are looking at two parallel tracks: what the host CPU thread is doing, and what the device is doing, moment to moment. Dispatch overhead shows up as a specific, recognisable shape. Between two kernel executions on the device track, there is a gap. On the host track, during that gap, the CPU is active — inside the driver, preparing the next launch. If those gaps are small relative to kernel duration, the host is keeping up. If the gaps are large, or if you see the device sitting idle while the host thread grinds through launch preparation, the host is the pacing item. On a RISC-V host with a less-optimised driver path, per-launch host time is exactly the thing that can inflate. Transfer-bound patterns look different: the memcpy operations on the timeline are long relative to the kernels they feed, and increasing batch size makes transfer dominate rather than amortise. Distinguishing dispatch-bound from transfer-bound from compute-bound is the whole game, and a single utilisation percentage cannot do it — this is why we frame profiling in terms of three pillars of observability applied to GPU utilisation rather than one headline metric. How mature is the GPU software stack on RISC-V, and how does that affect overhead? Honestly: it is uneven, and that unevenness is the actionable fact. The kernels themselves — the compiled device code that runs on the GPU — are largely host-agnostic once they are on the device. What varies by host is the userspace and driver stack: the CUDA runtime or ROCm equivalent, the libraries that stage transfers, the launch machinery. On x86 that stack has had years of platform-specific hardening. On RISC-V it is newer, and some of it may run through translation or compatibility layers rather than native builds. The practical consequence is that per-launch and per-transfer overhead is a moving target on RISC-V in a way it is not on x86. A workload profiled today may improve substantially as the vendor stack matures — but you cannot assume that away, and you certainly cannot conclude “the kernel is the constraint” without measuring the host path first. This maturity gap is also where the infrastructure-cost argument for RISC-V meets reality: an open ISA changes the economics, but only once profiling confirms the accelerator path is actually being fed. We treat the software stack as a first-class part of the executor, not a given. The unit of performance is the host, the driver, and the device together — never the kernel in isolation. When should I profile the host-to-device path before assuming compute-bound? The short version: always, and especially on a RISC-V server. But there is a decision rubric that makes it concrete. Profile the full host-to-device path before concluding a workload is compute-bound when any of the following hold — and on an emerging host architecture, treat the bar as lower: The workload launches many small kernels rather than a few large ones (dispatch-sensitive). Throughput does not scale the way FLOP budgets predict as you add batch or concurrency. The host CPU is non-x86, or the GPU driver stack on this host is new to you. Utilisation is high but wall-clock time is worse than a comparable x86 host. You are about to spend engineering days re-tuning a kernel based on a utilisation counter alone. The payoff is measured in time-to-insight. A few hours with a timeline profiler that separates host stall, transfer, and compute can save weeks spent optimising a kernel that was never the constraint — and when the real bottleneck is a host-bound or transfer-bound path, fixing it typically yields a larger speedup than kernel re-tuning would have, because you are removing a wall rather than shaving a corner. That is the same measurable outcome that governs the parent discipline: target the real constraint, not the assumed one. FAQ How does a RISC-V server actually work? A RISC-V server is a datacentre host built around a RISC-V CPU — an open instruction set architecture — running Linux and hosting accelerators like any other server. In practice the host still does all the non-GPU work of a workload: PCIe transfers, kernel dispatch, interrupt handling, and memory management. The ISA of that host shapes how fast and predictable this host work is, which is why a RISC-V host cannot be treated as interchangeable plumbing. How does a RISC-V host CPU differ from x86 when driving a GPU workload? The difference that matters is usually not raw compute but the maturity of the driver and userspace stack, host-side dispatch behaviour, and PCIe/NUMA tuning. On mature x86 hosts these host-side steps are fast and invisible on a trace; on RISC-V they can be less optimised or run through compatibility layers, adding per-launch and per-transfer overhead. The host becomes a legible variable rather than a given. Can a RISC-V host become the bottleneck even when GPU utilisation looks high? Yes. GPU utilisation only reports that work was scheduled on the device during a sampling window, not that the work was useful or saturating. A workload can show 90%+ utilisation while the GPU stalls waiting for the host to stage the next transfer or issue the next launch. On a slower or less-optimised host, that gap widens, so the workload is host-bound even though the counter reads compute-bound. What does a profiler trace look like on a RISC-V host, and where does host-side dispatch overhead show up? A timeline profiler such as Nsight Systems puts host and device on the same clock as two parallel tracks. Dispatch overhead appears as gaps between kernels on the device track while the host CPU is active preparing the next launch. Large gaps, or an idle device while the host grinds through launch preparation, indicate the host is pacing the workload — a pattern more likely on a RISC-V host with a less-optimised driver path. How mature is the GPU/accelerator software stack on RISC-V, and how does that affect transfer and driver overhead? It is uneven. The device kernels are largely host-agnostic once on the GPU, but the userspace and driver stack — runtime, transfer libraries, launch machinery — has had far less platform-specific hardening on RISC-V than on x86, and some may run through translation layers. That makes per-launch and per-transfer overhead a moving target, so you cannot assume it away when concluding a workload is compute-bound. When should I profile the host-to-device path before assuming a workload on a RISC-V server is compute-bound? Whenever the workload launches many small kernels, throughput does not scale with FLOP budgets, the host is non-x86 or the driver stack is new to you, utilisation is high but wall-clock time trails a comparable x86 host, or you are about to spend days re-tuning a kernel on the strength of a utilisation counter. On an emerging host architecture, treat that bar as lower and profile the full path first. The bottleneck you assume is the one you never measure Declaring a workload compute-bound because a utilisation counter is high — without ever looking at the host-to-device timeline — is engineering by superstition. It is a cheap habit on hardened x86 and an expensive one on a RISC-V server, where the host is exactly the part still settling. A GPU Performance Audit exists to close that gap: it profiles the full host-to-device path so a non-x86 host is captured as a candidate bottleneck rather than assumed away. The question to carry into your next profiling session is not “how fast is the kernel” but “on this host, this driver, and this device together, where does the time actually go?”
