“LLMOps” gets pitched as a fresh discipline that requires its own toolchain. In practice, most of the lifecycle reuses the MLOps primitives a team already runs, while a smaller — but higher-stakes — subset genuinely diverges: prompt management, eval-set drift, retrieval freshness, and cost-per-token monitoring. Teams that buy LLMOps wholesale duplicate spend; teams that pretend LLMs are “just another model” miss the eval and cost failure modes. The cleaner way to think about this is divergence-first — name the LLM-specific stages, instrument only those, and reuse the rest. We see this pattern regularly in platform conversations. A data team has a working MLOps stack — feature store, training pipelines, model registry, monitoring — and a generative-AI initiative shows up demanding “an LLMOps platform”. The right question is not “which platform” but “which stages of our existing lifecycle actually break when the model is a frontier LLM?” Where the LLM lifecycle reuses MLOps primitives The instinct to declare LLMOps a separate discipline obscures how much of the stack is unchanged. Classical MLOps covers data ingestion, transformation, lineage, model registry, deployment, autoscaling, request logging, and post-hoc monitoring of input/output distributions. None of these become obsolete when the model is an LLM. A retrieval-augmented chat endpoint still needs structured logging, version pinning, blue/green deployments, and a registry entry that ties the running artefact back to the code and prompt set that produced it. The reuse is most obvious in three places: Serving and autoscaling. A vLLM or TGI deployment is, infrastructurally, an inference service. The same Kubernetes patterns, the same GPU node pools, the same NCCL and CUDA assumptions apply. There is no LLM-specific reason to rebuild the serving substrate. Lineage and registry. Every production LLM call traces back to a base model version, a fine-tune (if any), a prompt template, and a retrieval index. The registry pattern from MLflow or a vendor equivalent extends cleanly — prompts and indexes become first-class registered artefacts alongside model weights. Request-level observability. Latency histograms, error rates, and traffic shaping are model-agnostic. The MLOps observability stack covers them. Treating these as new because the model is new is the duplication trap. Vendors quoting six-figure LLMOps SKUs to teams that already own MLflow, Airflow, and a Kubernetes inference layer are charging twice for the same primitives. Where it genuinely diverges A smaller set of lifecycle stages does not have a classical MLOps analogue, or has one so weak that it functionally needs to be rebuilt. This is where the actual investment belongs. Prompt management as a first-class artefact A prompt template is code that runs at inference time but is rarely treated like code. In a mature LLMOps setup the prompt has a version, an owner, an associated eval set, and a deployment record. Changing a prompt is a release event — diffable, reviewable, and rollback-capable. Classical MLOps has no equivalent because classical models do not have a free-text control surface that downstream teams can edit without retraining. Tools like LangSmith, Langfuse, and Humanloop occupy this gap; in our experience the gap is real and most teams underinvest here for the first six months of production use. Eval-set drift and regression testing Classical models are evaluated on a held-out test set with stable metrics — accuracy, AUC, F1. LLMs are evaluated on tasks where the right answer is often a distribution of acceptable responses, and where the eval set itself drifts as user behaviour shifts. The operationally relevant question is not “what’s the BLEU score” but “what share of production traffic is covered by an automated regression test, and how quickly does the eval suite catch a behaviour regression after a base-model swap or prompt edit?” This is an observed pattern across our LLM engagements: teams that can answer those two questions in numbers have far shorter incident loops than teams that rely on spot-checks. Cost-per-token monitoring A classical model has predictable inference cost — a function of input size, hardware, and batch. An LLM has cost that scales with both input and output tokens, varies by route (which model tier handles the request), and is sensitive to retrieval payload size. The cost surface is dynamic in a way classical MLOps cost dashboards do not capture. Cost-per-query trajectory, broken down by user segment and route, is the operationally relevant measure — not aggregate monthly spend. Retrieval freshness If the system is retrieval-augmented, the retrieval index is part of the model from the user’s perspective. Stale embeddings produce wrong answers without producing a model-level error. Classical drift detection looks at input/output distributions; retrieval freshness needs index-level instrumentation — last-update timestamps per shard, query-to-document recency histograms, and re-embedding cadence tied to source data change rates. What does an LLMOps stack look like that does not duplicate MLOps? The decision rubric below is the one we use when a platform team asks whether to extend or replace. It is deliberately divergence-first. Lifecycle stage Classical MLOps covers it? Action Data ingestion / ETL Yes Reuse Model registry (base + fine-tune weights) Yes Reuse, extend schema Prompt versioning and review No Add a prompt registry Retrieval index lifecycle Partial Add index-freshness telemetry Eval-set management and regression tests Weak Build LLM-specific eval harness Serving infra (GPU pools, autoscaling) Yes Reuse Request/response logging Yes Reuse, add token-level fields Cost monitoring Partial Add per-route, per-token attribution Output safety and red-teaming No Build or buy targeted tooling Incident response Yes Reuse runbooks, extend with prompt rollback An observed pattern across our engagements (planning heuristic, not a benchmarked rate): roughly 60–70% of a working MLOps stack carries over without modification, 15–20% needs a targeted extension, and only the remaining 15–20% is genuinely new build. Teams that scope their LLMOps investment to that last slice ship in weeks. Teams that re-platform end-to-end ship in quarters and explain the duplication for years. When is a separate LLMOps platform worth the spend? Three conditions that, in our experience, justify a separate platform investment rather than an extension: The existing MLOps stack does not exist or is shallow. If there is no registry, no lineage, and no deployment automation, the team is not extending MLOps — they are buying their first ops platform. Buying one designed for LLMs is reasonable in that case. Prompt and eval workflows are the bottleneck. If the engineering team is shipping prompt changes faster than the data team can register them, a purpose-built prompt-and-eval platform pays for itself in incident reduction. Multi-vendor model routing is in scope. If the production system routes between OpenAI, Anthropic, a self-hosted Llama variant, and a fine-tune, the routing, cost-attribution, and fallback logic is heavy enough to warrant dedicated tooling. Outside those conditions, the answer is usually to extend the existing platform with a prompt registry, an eval harness, and token-level cost telemetry — and to keep the rest of the MLOps stack intact. A note on what gets called “LLMOps” The term is doing two jobs at once in the market. One job is to name the genuinely new lifecycle stages — prompt management, eval drift, retrieval freshness, cost-per-token. The other is to sell a parallel platform. Teams that separate those two meanings before they start vendor conversations end up with an instrumented LLM lifecycle. Teams that conflate them end up with two platforms doing the same work. This connects directly to engagement scoping. When we work with a client on operationalising LLMs, the R&D engagement plan names which lifecycle stages the team owns versus which are vendor-managed, and the divergence map above is an input to that decision — not an afterthought. FAQ Where does the LLM lifecycle genuinely diverge from the classical ML lifecycle, and where does it reuse the same primitives? Serving infrastructure, registry, lineage, and request-level observability reuse classical MLOps primitives. The genuine divergence is in prompt management as a versioned artefact, eval-set drift detection, retrieval index freshness, and cost-per-token monitoring. Roughly 60–70% of a working MLOps stack carries over without modification. What does an LLMOps stack look like that does not duplicate the underlying MLOps stack? It extends rather than replaces. Add a prompt registry, an LLM-specific eval harness, index-freshness telemetry, and per-route token-cost attribution on top of the existing registry, lineage, serving, and monitoring layers. The duplication trap is paying twice for the primitives the MLOps stack already provides. How is eval-set drift detected and acted on for production LLMs? By tracking the share of production traffic covered by automated regression tests, and the time from a base-model swap or prompt edit to the eval suite catching a behaviour regression. Eval sets themselves need versioning and periodic refresh against current production traffic, because user behaviour shifts the distribution of inputs. Which cost controls actually constrain LLM spend in 2026 vs which are theoretical? The constraints that bind in practice are per-route token attribution, prompt-length budgets enforced at the gateway, and retrieval-payload size caps. Caching helps where traffic is repeat-heavy. Theoretical controls — like fine-tuning a smaller model to replace a frontier call — work for narrow tasks but rarely deliver the cost reduction the business case assumed. How is prompt management treated as a first-class artefact in LLMOps? The prompt has a version, an owner, an associated eval set, and a deployment record. Changes are reviewed and rollback-capable. The prompt registry sits alongside the model registry, not inside application code. When is a separate LLMOps platform worth the spend vs extending the existing MLOps platform? When the existing MLOps stack is shallow or absent, when prompt-and-eval workflows are the operational bottleneck, or when multi-vendor model routing is in scope. Otherwise the answer is extension — add the prompt registry, eval harness, and token-cost telemetry on top of what already exists. Continue reading For the foundational view of why ops discipline matters at all, see our introduction to MLOps and the longer take on MLOps for organisations that have never operationalised a model. For the model-size axis of the same conversation, see small and large language models. The lifecycle divergence map sketched here is one of the inputs to the R&D engagement plan we use when scoping LLM operationalisation work. The other inputs — cost trajectory targets, eval-coverage targets, and ownership boundaries between client and vendor — are where the real conversation starts.
