“DeepSeek runs on Intel now” is true, and it is also the sentence that gets an inference budget into trouble. The claim is real — Xeon CPUs, Arc GPUs, and Gaudi accelerators can all serve DeepSeek. What it does not tell you is whether moving your deployment onto Intel silicon will lower the price you pay per request, or just move that same cost sideways into a different line item. Those are two very different outcomes, and only one of them shows up on the invoice. The trap is subtle because the words sound like a hardware decision. “Intel DeepSeek” reads as a swap: take the model you already serve, put it on Xeon or Gaudi, watch cost-per-request fall. In practice, the hardware is one variable in a serving path that also includes the runtime backend, the quantisation format, the batch size, and the routing logic in front of the model. A migration that ignores those variables is not a cost optimisation. It is a bet that the bottleneck was hardware — and that bet is frequently wrong. What should you know about Intel DeepSeek in practice? There is no special “Intel DeepSeek” build. DeepSeek is a family of open-weight models — DeepSeek-V3, DeepSeek-R1, and their distillations — and “running it on Intel” means compiling and serving those weights through Intel’s software stack on Intel hardware. The phrase entered circulation because Intel publicly demonstrated DeepSeek inference across its product lines, and because open weights make the port legally and technically straightforward. What it means in practice is a serving-path choice. You pick a hardware target (Xeon, Arc, or Gaudi), a runtime that knows how to schedule the model on that target (OpenVINO, IPEX-LLM, oneDNN under the hood, or the Gaudi software suite), and a numerical format the runtime will execute in (BF16, INT8, INT4). The combination — not the chip alone — determines throughput, latency, and therefore cost-per-request. Treating “Intel” as the whole answer is like saying a car is fast because it has an engine; the tuning matters more than the label. We see the same pattern whenever a team reads a vendor demo as a procurement instruction. The demo proves the model can run. It says almost nothing about whether it runs cheaper than what you have under your traffic shape. Which Intel platforms can run DeepSeek inference — Xeon, Arc, or Gaudi? All three can, and they sit at different points on the cost-latency-scale curve. Choosing among them is the first real decision, and it depends on model size, expected concurrency, and how tight your p95 latency budget is. Intel serving targets for DeepSeek — a decision table Target Best fit Runtime path What it wins on What it costs you Xeon CPU Small/distilled DeepSeek, low concurrency, batch or async jobs IPEX-LLM / OpenVINO on oneDNN No accelerator to provision; uses hardware you likely already have; AMX INT8 acceleration Token throughput ceiling is lower; large models get memory-bandwidth-bound fast Arc GPU Mid-size models, moderate concurrency, latency-sensitive single-node serving OpenVINO / IPEX-LLM with SYCL Discrete GPU throughput at lower entry cost than datacenter parts Ecosystem maturity lags CUDA; kernel coverage varies by model op Gaudi accelerator Full DeepSeek-V3/R1 at scale, high concurrency, throughput-oriented serving Gaudi software suite (SynapseAI/Intel Gaudi runtime) Purpose-built for transformer inference at scale; strong throughput-per-dollar in the right regime Requires re-provisioning; the migration itself is a real capital and engineering cost The table is a starting filter, not a verdict. A distilled DeepSeek model handling a few requests per second may run acceptably on Xeon you already own — which makes the migration cost effectively zero. A high-concurrency chat workload serving DeepSeek-V3 is a Gaudi conversation, but only if profiling shows your current path is genuinely saturated. This is the same profiling discipline that applies when reading llama.cpp benchmark numbers before you port an LLM: the hardware shortlist is downstream of measurement, not upstream of it. What runtime stack do you need to serve DeepSeek on Intel? The runtime is where most of the cost delta actually lives, and it is the part teams under-weight. On Intel, the practical options cluster around a few components. OpenVINO is Intel’s cross-hardware inference runtime. It ingests a model (often via an intermediate representation converted from PyTorch or ONNX), applies graph optimisations, and targets CPU, Arc GPU, or NPU. It handles the compilation and quantisation-aware execution that makes CPU serving viable. IPEX-LLM (Intel Extension for PyTorch, LLM path) keeps you closer to a PyTorch-native workflow while pulling in Intel-specific kernels and low-bit quantisation. For teams that want minimal code change from an existing PyTorch serving setup, this is often the lower-friction entry point. oneDNN is the low-level primitive library — the fused kernels for GEMM, attention, and activation — that OpenVINO and IPEX-LLM lean on underneath. You rarely call it directly, but it is why AMX instructions on recent Xeon deliver usable INT8 throughput. If you want the broader picture of where these compilation layers sit in a serving path, our explainer on what machine learning compilers do in the inference serving path maps the same territory. Gaudi runtime is a separate world: the Gaudi software suite has its own graph compiler and execution model, and porting to it is closer to a platform migration than a library swap. The point is that the same DeepSeek weights, on the same Xeon box, can differ by a large margin in tokens-per-second depending on whether they run through a naive PyTorch loop or through OpenVINO with INT8 quantisation and proper batching. In configurations we have profiled, the runtime and quantisation choices frequently account for more of the throughput than the choice of chip within a tier — an observed pattern across inference-cost engagements, not a published benchmark. Does moving DeepSeek to Intel hardware actually lower cost-per-request? Sometimes. And the only honest way to know is to measure your current serving path first, because cost-per-request is a system property, not a hardware property. Cost-per-request is roughly your hourly infrastructure cost divided by the requests that hardware serves per hour at your target latency. A hardware swap changes the numerator (the hourly rate) and can change the denominator (throughput), but it does nothing about three levers that live in software: Batching. An under-batched serving path leaves the accelerator idle between requests. Fix the batching and throughput rises on the hardware you already have — no migration required. The unit-economics framework for AI infrastructure treats cost-per-request as the KPI precisely because it exposes this. Routing. If you send every request to the full DeepSeek-V3 when a distilled variant would answer most of them correctly, the model, not the chip, is the cost. Routing addresses that regardless of vendor. Quantisation. An un-quantised BF16 deployment on any hardware costs more than a well-validated INT8 or INT4 one. If you are comparing an FP16 GPU path against an INT8 Intel path, you may be measuring the quantisation, not the silicon. Here is the failure that an inference cost audit is designed to catch: a team migrates DeepSeek to Intel, sees cost-per-request drop, and attributes the win to the hardware — when the real driver was that the migration forced them to re-batch and quantise, changes that would have delivered most of the savings on their original stack. The Intel move looked like the cause. It was the occasion. A quick diagnostic before you commit to Intel Ask these in order. If you cannot answer the first three from current telemetry, you are not ready to judge a hardware migration. What is my current cost-per-request and p95 latency, measured under production-shaped traffic? What is my accelerator/CPU utilisation at steady state? (Low utilisation means a batching or routing problem, not a hardware problem.) Am I running quantised? If not, what does INT8/INT4 do to cost and quality on my current stack? Only now: does the Intel target beat the tuned current stack on cost-per-request at equal p95 and equal output quality? How do quantisation and batching affect cost independently of the hardware? These are the two levers that most often get miscredited to a hardware change, so they deserve isolating. Quantisation reduces the numerical precision of weights and activations — BF16 to INT8, sometimes INT4 — which shrinks memory footprint and raises the effective throughput of the same compute. On Intel specifically, AMX on Xeon and the low-bit paths in OpenVINO and IPEX-LLM are built to exploit this. But quantisation is a quality decision as much as a cost one: aggressive INT4 can degrade DeepSeek’s reasoning outputs in ways that only show up on task-level evaluation, not on a latency graph. Measure both. Batching amortises fixed per-step overhead across more requests. Continuous or dynamic batching in a modern serving runtime can lift throughput several-fold over naive one-request-at-a-time serving. Crucially, this lever exists on your current GPU stack too — which is why an apples-to-apples Intel comparison must tune batching on both sides. Comparing an un-batched GPU baseline against a batched Intel path is a rigged benchmark, and the wrong conclusion is expensive. The discipline here mirrors what we describe in running DeepSeek on Intel hardware and what profiling tells you about inference cost: the numbers only mean something when the two paths are tuned to the same standard before you compare them. How do you benchmark a DeepSeek-on-Intel deployment against your current path? Benchmark the system, not the chip. A defensible comparison holds output quality and p95 latency fixed, then measures cost-per-request on each path — each tuned to its best achievable batching and quantisation. Concretely: replay a representative sample of production traffic (real prompt-length distribution, real concurrency) against both serving paths. Record tokens-per-second, p95 and p99 latency, hardware utilisation, and hourly cost. Validate output quality on a task suite so a quantisation-driven quality regression cannot masquerade as a cost win. This is the same methodology used to benchmark GPU serving paths, and it applies unchanged across Intel and accelerator backends — the profiling approach for serving DeepSeek on SGLang and what profiling reveals transfers directly. For the audit itself, our [AI Inference Cost Audit](Inference Cost-Cut Pack) exists to run exactly this comparison — profiling whether an Intel/DeepSeek serving path actually lowers cost-per-request, or whether the bottleneck sits in batching, routing, or quantisation regardless of vendor. When is an Intel migration the right lever versus optimising the existing runtime first? Optimise the existing runtime first — almost always. Batching, quantisation, and routing are cheaper to change than hardware, they carry no migration risk, and they establish the tuned baseline you need before any hardware comparison is meaningful. Skipping them means you are comparing an Intel path against a strawman. The Intel migration becomes the right lever when your current stack is genuinely saturated — high utilisation, tuned batching, validated quantisation — and the numbers still say the price-per-request or the throughput ceiling is the problem. Then a Gaudi or Arc target that offers better throughput-per-dollar in your regime is a measured decision, not a hopeful one. It is also the right lever when procurement or sovereignty constraints make the vendor itself the requirement, in which case the question shifts from “is it cheaper” to “how do I make it as cheap as possible on this platform.” FAQ What does working with Intel DeepSeek involve in practice? There is no dedicated “Intel DeepSeek” model — it means serving DeepSeek’s open weights through Intel’s software stack (OpenVINO, IPEX-LLM, oneDNN, or the Gaudi runtime) on Intel hardware. In practice it is a serving-path decision: the hardware target, runtime backend, and quantisation format together determine throughput and cost, not the chip alone. Which Intel platforms can run DeepSeek inference — Xeon CPUs, Arc GPUs, or Gaudi accelerators? All three can. Xeon suits small or distilled models at low concurrency and often uses hardware you already own; Arc suits mid-size latency-sensitive single-node serving; Gaudi is built for full DeepSeek-V3/R1 at high concurrency and scale. The right choice depends on model size, concurrency, and p95 latency budget — established by profiling, not by the vendor demo. What runtime stack do you need to serve DeepSeek on Intel? OpenVINO is Intel’s cross-hardware inference runtime; IPEX-LLM keeps a PyTorch-native workflow with Intel kernels and low-bit quantisation; oneDNN provides the fused primitives both lean on; and the Gaudi software suite is a separate compiler and execution model. The runtime and quantisation choices frequently drive more of the throughput than the chip choice within a tier. Does moving DeepSeek to Intel hardware actually lower cost-per-request, or just shift it? Sometimes it lowers cost, but only measurement tells you which. Cost-per-request is a system property: batching, routing, and quantisation live in software and can deliver most of the savings on your current stack. A common failure is crediting the hardware for a win that the migration’s re-batching and quantisation actually produced. How do quantisation and batching affect DeepSeek inference cost on Intel independently of the hardware? Quantisation (BF16 to INT8/INT4) shrinks memory footprint and raises effective throughput, and Intel’s AMX and OpenVINO low-bit paths exploit it — but aggressive quantisation can degrade output quality, so measure both cost and task accuracy. Batching amortises per-step overhead and exists on your current stack too, so any fair Intel comparison must tune batching on both sides. How do you benchmark a DeepSeek-on-Intel deployment against your current serving path before committing? Replay production-shaped traffic against both paths, each tuned to its best batching and quantisation, and hold output quality and p95 latency fixed while measuring cost-per-request, utilisation, and hourly cost. Validate output quality on a task suite so a quantisation-driven regression cannot masquerade as a cost win. When is an Intel migration the right lever versus optimising the existing runtime first? Optimise the existing runtime first in almost every case — batching, routing, and quantisation are cheaper, carry no migration risk, and establish the tuned baseline any hardware comparison needs. Migrate to Intel when the current stack is genuinely saturated and the tuned numbers still point to hardware, or when procurement or sovereignty constraints make the platform itself the requirement. The failure mode worth naming plainly: an Intel migration that skips profiling is not a cost decision, it is a hardware guess wearing a cost-decision’s clothes. If your DeepSeek serving path has never been profiled at the level of cost-per-request against a tuned baseline, the first question is not which Intel chip — it is which lever, and whether the one you are about to buy is the one that was actually stuck.
