“We need three months to figure out how to get DeepSeek into production.” That sentence is where a serving project quietly becomes a research project — and where the budget stops having a bottom. Running DeepSeek behind SGLang is a serving-infrastructure decision with known methods and predictable behaviour on bounded hardware. It is an engineering task with deliverables, milestones, and acceptance criteria. Treating it as an open-ended investigation is the most common way these projects overrun. The confusion is understandable. DeepSeek is a large, capable model family, SGLang is a serving runtime with its own vocabulary, and the two together sit close to the frontier of open-weight inference. Frontier-adjacent work feels like research. But feeling like research and being research are different things, and the difference is exactly what determines whether you can put a number on the work before you start. How does SGLang serve DeepSeek, and what does that mean in practice? SGLang is an inference-serving runtime. It takes a model — DeepSeek-V3, DeepSeek-R1, or one of the distilled variants — loads its weights onto GPUs, and answers requests with generated tokens. What separates it from a naive model.generate() loop is a set of mechanisms designed to keep GPUs busy and avoid recomputing work that has already been done. The central one is RadixAttention. When many requests share a prefix — a system prompt, a few-shot preamble, a retrieved document that several queries reference — RadixAttention stores the corresponding KV-cache entries in a shared radix tree and reuses them instead of recomputing attention over the same tokens for every request. For workloads with heavy prefix overlap, this is the difference between a runtime that scales and one that melts under concurrency. SGLang also provides structured-output generation (constraining decoding to valid JSON or a grammar), tensor parallelism to shard a model that does not fit on one GPU, and continuous batching so that new requests slot into the decode loop without waiting for a full batch to finish. None of these mechanisms is speculative. They are documented, benchmarkable, and have known behaviour on known hardware. That is the whole point. When you serve DeepSeek on SGLang, you are assembling components with predictable properties — not inventing them. The same framing applies whether you are reasoning about how SGLang and DeepSeek inference works and what to profile or about what the runtime choice costs and saves on DeepSeek-V3. The line that actually matters: engineering versus research Here is the distinction that governs how the project should be scoped. Ask what your uncertainty is about. If your uncertainty is about the serving stack — can I hit the throughput I need, will the KV cache fit, does tensor parallelism across four GPUs give me acceptable latency — that is engineering. It is uncertain in the way that any unmeasured system is uncertain, but the method for resolving it is known: configure, benchmark, tune, re-benchmark. You can bound the work because the questions have answers you can measure your way to. If your uncertainty is about whether DeepSeek can meet an accuracy bar on your data that has never been demonstrated — can it extract the right fields from your contracts, can it hold up on your domain’s jargon, does its reasoning survive your edge cases — that is research. The method is not “tune the runtime.” It is investigation: build an evaluation set, run the model against it, decide whether the results clear the bar. Until that question is answered, you cannot honestly scope the serving work, because you do not yet know if there is anything to serve. Misclassifying the second as the first is where serving projects overrun. A team budgets six weeks of “SGLang engineering” and spends four of them discovering that DeepSeek’s answers are not good enough on their documents — which was never a runtime problem. Whether DeepSeek clears a per-use-case accuracy bar is a generative-AI feasibility question, and it belongs on the research side of the line; our approach to generative AI feasibility treats that as a separate, prior investigation with its own gate. Resolve it first, then the serving task becomes pure engineering. A decision rubric: is this task engineering or research? Use this before you write a project plan. Score each row; if every uncertainty lands in the left column, you have an engineering task you can put milestones on. Question you are trying to answer Class How you resolve it Can SGLang serve DeepSeek at N tokens/sec under my concurrency profile? Engineering Benchmark against a baseline; tune batching and TP degree Will the KV cache and weights fit in my GPU memory budget? Engineering Compute memory footprint; measure under load; adjust quantization Does RadixAttention give useful prefix reuse for my traffic shape? Engineering Profile with representative prompts; measure cache hit rate Is time-to-first-token acceptable at my target concurrency? Engineering Load-test; adjust prefill/decode allocation Does DeepSeek clear my accuracy bar on my domain data? Research Build an eval set; run it; decide before scoping serving Can the model do a task no one has demonstrated it can do? Research Investigate feasibility first; do not budget serving yet The rows in the left column all share a property: the answer is a number you can measure before committing full budget. That is what makes them engineering. The two research rows share the opposite property — you cannot benchmark your way to the answer without first doing an investigation whose outcome could be “no.” What should I benchmark before committing budget? If the work is engineering, it has targets. Three matter most, and all three are measurable against a baseline before you commit to the full build. Throughput — tokens per second. Under a defined concurrency profile (say, 32 concurrent requests with a given prompt-length distribution), how many output tokens per second does the SGLang-served DeepSeek endpoint sustain? This is the number that determines whether one GPU node covers your traffic or whether you need four. It is a benchmark-class figure: reproducible, tied to a named configuration, and comparable across runtime settings. Time-to-first-token latency. For interactive workloads, the tail latency to the first token often matters more than raw throughput. Prefill-heavy requests — long retrieved contexts — can push TTFT up sharply. Splitting the prefill and decode phases across resources is one lever here; the mechanics of prefill/decode disaggregation cutting inference latency show why the two phases have different resource profiles and why serving them together can leave both under-served. GPU-memory utilisation under a defined concurrency profile. The KV cache grows with concurrency and context length. A configuration that runs fine at 8 concurrent requests can OOM at 40. Measuring memory headroom under your target load — not your average load — tells you whether the deployment is stable or one traffic spike away from an outage. These three numbers, measured against a baseline, are what let a buyer fund a deliverable with acceptance criteria instead of an open-ended investigation with no termination point. “Sustain 1,200 tokens/sec at 32 concurrent requests with p95 TTFT under 800 ms on a 4×H100 node” is a spec. “Figure out DeepSeek serving” is not. When does a deployment cross into research? The crossing is not about model size or novelty of the runtime. It is about whether a required outcome has been demonstrated. Serving a model that is known to do the job is engineering, however large the model. Asking a model to do something no one has shown it can do — on your data, at your quality bar — is research, even if the serving stack is trivial. A useful test: could you write an acceptance criterion today and know, in principle, how to measure whether it was met? For “1,200 tokens/sec at 32 concurrency,” yes — you know the measurement even before you run it. For “extracts the correct clause from every contract type in our portfolio,” the honest answer is often “we don’t yet know if the model can, and we won’t know until we build an eval set and run it.” That second case is research, and it must be resolved before the serving work is scoped as engineering. This is precisely the boundary an A5-style research-boundary assessment exists to draw: it puts the serving of DeepSeek on the engineering side and any unproven accuracy question on the research side, so the two are funded differently and neither contaminates the other’s schedule. How do I scope a SGLang-on-DeepSeek project with milestones? Once the accuracy question is resolved — or confirmed to be a non-issue because the model is already known to clear the bar — the serving work decomposes cleanly into stages with checkpoints. A workable arc looks like this. First, establish a baseline: DeepSeek on SGLang with default settings, measured for throughput, TTFT, and memory under a representative load. Second, tune the known levers — tensor-parallel degree, batch settings, quantization level, RadixAttention configuration — and re-measure against the baseline. Third, validate stability under the concurrency profile you actually expect, including headroom for spikes. Each stage ends in a number, and each number is checked against the acceptance criteria agreed up front. If a stage misses, you pivot with evidence rather than sentiment — the same intermediate-checkpoint discipline that governs how a POC is scoped with checkpoints and pivot evaluation. This is what makes it fundable. Every milestone has a deliverable, every deliverable has a measurable acceptance criterion, and the whole engagement has a defined endpoint. In our experience, that is the single largest difference between DeepSeek serving projects that land on schedule and those that drift — not the choice of runtime, but whether the work was correctly classified as engineering before anyone opened a terminal. FAQ How does SGLang serve DeepSeek, and what does it mean in practice? SGLang is an inference-serving runtime that loads DeepSeek’s weights onto GPUs and answers requests with generated tokens, using mechanisms like RadixAttention, continuous batching, and tensor parallelism to keep GPUs busy. In practice it means you are assembling components with documented, benchmarkable behaviour rather than inventing serving techniques from scratch. Which parts of running DeepSeek on SGLang are engineering tasks with known methods, and which parts require research first? Anything about the serving stack — throughput, KV-cache fit, latency, tensor-parallel configuration — is engineering, because the method for resolving the uncertainty is configure-benchmark-tune. Whether DeepSeek clears an accuracy bar on your data that has never been demonstrated is research, and it must be resolved by investigation before the serving work can be honestly scoped. What does SGLang provide, and how does that make serving predictable? SGLang provides RadixAttention (shared-prefix KV-cache reuse via a radix tree), structured-output generation, tensor parallelism for sharding models across GPUs, and continuous batching. These mechanisms have documented, measurable behaviour on known hardware, so serving becomes an assembly of predictable components rather than an open investigation. What throughput, latency, and GPU-memory targets should I benchmark before committing budget? Measure three numbers against a baseline: sustained tokens-per-second under a defined concurrency profile, time-to-first-token latency at target concurrency, and GPU-memory utilisation under your peak load rather than your average. These let a buyer fund a deliverable with acceptance criteria instead of an open-ended investigation. When does a DeepSeek deployment cross from an engineering task into a research question? It crosses when a required outcome has not been demonstrated — for example, asking the model to do something no one has shown it can do on your data at your quality bar. The test: could you write an acceptance criterion today and know how to measure whether it was met? If not, that part is research and must be resolved first. How do I scope a SGLang-on-DeepSeek project with milestones and acceptance criteria? Once the accuracy question is settled, decompose the work into stages: establish a baseline, tune the known levers and re-measure, then validate stability under your expected concurrency profile. Each stage ends in a measured number checked against acceptance criteria agreed up front, giving the engagement a defined endpoint rather than open-ended spend. The runtime is not the risk. The risk is spending engineering time on a question the runtime cannot answer — which is why the first deliverable on any DeepSeek serving engagement should be the line itself: what here is benchmarkable, and what has to be investigated before we agree the benchmark even matters.
