A team sizes an LLM serving deployment from a spreadsheet of word counts. They assume one word equals one token, provision KV-cache and batch memory accordingly, and set a cost-per-request budget. Then the tokeniser hands them roughly 30% more tokens than words on English prose — and far more on code and non-Latin scripts. Every downstream estimate, from cost per request to context-window headroom, was built on the wrong unit. That gap is the whole point of this article. The tokens-per-word ratio is not a rounding detail. It is a measurable input to your inference profile, and treating it as a one-to-one conversion is where cost, latency, and memory estimates quietly drift off by a fifth to nearly a half. What does average tokens per word mean in practice? Language models do not read words. They read tokens — sub-word fragments produced by a byte-pair encoding (BPE) tokeniser such as the one behind GPT models, or a SentencePiece tokeniser like the ones used by many open Llama-family and Mistral models. The tokeniser splits text into whatever units its vocabulary was trained to represent, and the model’s cost is denominated entirely in those units. The “average tokens per word” ratio is simply the token count of a piece of text divided by its word count. For typical English prose, that ratio sits close to 1.3 tokens per word — meaning a 1,000-word document becomes roughly 1,300 tokens before the model does any work (observed pattern across common OpenAI and Llama tokenisers; not a single benchmarked figure, since it varies by content). Common short words map to a single token. Longer or rarer words, inflected forms, and anything with unusual spelling get split into several. The ratio matters because everything you pay for and everything you provision is priced in tokens. Cloud inference APIs bill per token. KV-cache memory grows with the number of tokens held in context. Context windows are capped in tokens, not words. If you reason in words, you are reasoning in the wrong currency and converting at a made-up exchange rate. Why is the tokens-per-word ratio usually greater than one? A word-level vocabulary would need an entry for every word in every language, including every misspelling, every rare technical term, and every proper noun. That vocabulary would be enormous and would still fail on words it had never seen. Sub-word tokenisation solves this by keeping a fixed vocabulary — commonly on the order of 30,000 to 130,000 entries depending on the model — and representing anything outside it as a sequence of smaller pieces. The consequence is that the average word costs more than one token. Frequent words earn a dedicated slot; the long tail gets fragmented. A word like “the” is one token. A word like “tokenisation” may become three or four, and a hyphenated or camel-cased identifier can fragment further still. The more your text leans on rare vocabulary, the higher the ratio climbs. Two forces push the ratio around. Vocabulary coverage pushes it down: a tokeniser trained heavily on your content type will represent it more compactly. Content novelty pushes it up: unusual spellings, dense punctuation, numbers, and whitespace all fragment. This is the same reasoning we cover in GPT Token Count Explained: How Tokens Drive Inference Cost, which walks through how the raw token count — not word count — is the billable and memory-relevant quantity. How does the ratio vary across languages, code, and content type? The 1.3 figure is an English-prose anchor, and it is the wrong number for almost everything else. The ratio is a property of the interaction between your tokeniser and your content, so it moves substantially with content type. The table below shows illustrative ranges we use as planning heuristics before profiling a real workload. Treat them as order-of-magnitude framing, not benchmarks — the exact figures depend on the specific tokeniser and text sample. Tokens-per-word by content type (planning heuristics) Content type Approx. tokens per word Why it diverges English prose ~1.3 Common words map to single tokens; long tail fragments Technical / scientific English ~1.5–1.8 Dense rare vocabulary, units, identifiers Source code ~2–4 (per whitespace-word) Symbols, indentation, camelCase, and operators fragment heavily Non-Latin scripts (e.g. CJK) highly variable, often >2 Vocabulary coverage is thinner; characters may map to multiple tokens JSON / structured markup ~2–3 Braces, quotes, and keys tokenise as separate units Ranges are observed planning heuristics across common tokenisers, not published benchmarks. Measure your own. The practical takeaway: a code-generation product and a customer-support chat product with identical word volumes can have token volumes that differ by more than 2x. If both were sized from word counts using the same assumption, one of them is badly mis-provisioned. How do I measure the actual ratio for my model’s tokeniser? You do not estimate this. You measure it, because the tokeniser is deterministic and the measurement is cheap. Take a representative corpus. Pull a few thousand real requests (or realistic samples) from the actual workload — not a generic English paragraph. The corpus must match the content type you will serve. Run it through the exact tokeniser your model uses. For OpenAI-family models this is tiktoken; for Hugging Face models it is the model’s bundled tokeniser via the transformers library. Do not substitute a different tokeniser — vocabularies differ, and so do the ratios. Divide total tokens by total words. Use a consistent word-counting rule (whitespace split is fine as long as you apply it everywhere) and compute the aggregate ratio, not a per-sentence average. Segment by content type. If you serve mixed traffic — prose plus code, or multiple languages — compute the ratio per segment and weight by traffic share. A single blended number hides the segments that will hurt you. For the mechanics of the counting step, our walkthrough of how tiktoken counts tokens shows the encoding path in practice, and the token calculator for LLMs covers turning that count into a pre-deployment cost estimate. How does the ratio affect inference cost and context budgeting? This is where the abstract ratio becomes money and memory. Two concrete channels: Cost per request. APIs and self-hosted throughput budgets are both denominated in tokens. If you estimate cost from word counts assuming one-to-one, and the real ratio is 1.3, your per-request token count — and therefore your bill — is understated by 30% before you account for the model’s generated output tokens, which have their own ratio. In our experience, cost-per-1K-token estimates derived from raw word counts land 20–40% off once the true ratio and output length are folded in (observed pattern from workload-sizing engagements; not a benchmarked rate). That is the difference between a serving deployment that fits its budget and one that blows through it in the first month. KV-cache memory and context sizing. The KV-cache holds key and value tensors for every token in context, and its footprint scales linearly with token count. Size the cache from word counts and you under-provision GPU memory — which shows up as smaller feasible batch sizes, more frequent context evictions, or outright out-of-memory failures under load. Context windows are capped in tokens too: a “10,000-word document fits in a 16K context” assumption fails the moment the ratio pushes that document past 13,000 tokens. If KV-cache pressure is your real constraint, the reuse mechanics in RadixAttention Explained: KV-Cache Reuse and Inference Cost on GPU Clusters are the next lever, but they only help once your token accounting is honest. Getting the ratio right does not require profiling the whole serving stack first. It is a cheap, upfront correction that makes every subsequent estimate start from real token volumes. When we run a GPU performance and cost audit, accurate tokens-per-word ratios sharpen the workload characterisation — the audit profiles real behaviour, but it profiles it against token volumes that already reflect the tokeniser’s arithmetic rather than a word-count guess. Why can word-count-based cost estimates mislead a serving workload? Because the error is not random noise that averages out — it is a systematic, direction-consistent underestimate that compounds. Word counts always understate tokens (the ratio is greater than one), so every mis-sized quantity errs the same way: too little memory, too small a batch, too low a cost estimate. There is no offsetting overestimate elsewhere to cancel it. The failure also hides until load arrives. A single test request looks fine because the absolute token difference is small. It is only at production traffic — thousands of concurrent contexts filling KV-cache, sustained billing accumulating token by token — that a 30% systematic understatement becomes a capacity incident or a budget overrun. By then the provisioning decisions are baked into the deployment. The fix is discipline, not tooling: denominate every estimate in measured tokens from the start, segment by content type, and never let “words” stand in for “tokens” in a sizing calculation. FAQ What should you know about average tokens per word in practice? It is the token count of a text divided by its word count, using the exact tokeniser your model runs. Because models read sub-word tokens rather than words, this ratio is the conversion between the units humans count (words) and the units that drive cost, memory, and context limits (tokens). For English prose it sits near 1.3, so a 1,000-word document is roughly 1,300 tokens. Why is the tokens-per-word ratio usually greater than one, and what drives it up or down? Sub-word tokenisers keep a fixed vocabulary and split anything outside it into smaller pieces, so the average word costs more than one token. Frequent words get a single token; rare words, unusual spellings, numbers, and dense punctuation fragment into several. Good vocabulary coverage of your content pushes the ratio down; novel or symbol-heavy text pushes it up. How do average tokens per word vary across languages, code, and content type? Substantially — the 1.3 English-prose figure is the wrong anchor for most content. Technical English runs higher, source code often lands at two to four tokens per whitespace-word, and non-Latin scripts vary widely and frequently exceed two. Two products with identical word volumes but different content types can differ in token volume by more than 2x. How do I measure the actual tokens-per-word ratio for my model’s tokeniser? Take a representative sample of real requests, run it through the exact tokeniser your model uses (tiktoken for OpenAI-family models, the bundled Hugging Face tokeniser otherwise), and divide total tokens by total words. Segment by content type and weight by traffic share if you serve mixed workloads. It is a cheap, deterministic measurement — do not estimate it. How does the tokens-per-word ratio affect inference cost estimates and context-window budgeting? Cost and KV-cache memory both scale with token count, so a word-count estimate assuming one-to-one understates both. Cost-per-1K-token figures derived from word counts commonly land 20–40% off once the true ratio and output tokens are included. Context windows are token-capped too, so a document that “fits” by word count may overflow once tokenised. Why can word-count-based cost estimates mislead when profiling an LLM serving workload? Because the error is systematic, not random: word counts always understate tokens, so memory, batch size, and cost all err in the same direction and compound. The gap stays invisible on single test requests and only surfaces at production load, when it becomes a capacity incident or budget overrun. Denominating every estimate in measured tokens from the start removes the failure mode. The open question for any serving team is not whether the ratio matters — it always does — but which content segment in your traffic mix has the highest ratio, because that segment sets the ceiling on your memory and cost exposure. Measure the tail, not just the average, before you size the stack.
