Introduction Markov chains predict the next state of a process from the current state alone — no memory of the path taken to get there. That memorylessness is both their power and their ceiling. They named a mathematical idea after Andrey Markov more than a century ago, and the same idea still underwrites parts of modern AI: PageRank, hidden Markov models in speech, sampling chains inside diffusion training, certain reinforcement-learning formulations. What they no longer do — and this is the part most casual explainers get wrong — is generate high-quality text or images on their own. That work moved to transformers, GANs, and diffusion models years ago. In our experience reviewing client architectures, the assumption that “an AI chatbot uses a Markov chain to predict the next word” still surfaces regularly. It hasn’t been true for production-grade generative AI since the rise of attention-based language models. This article draws the line cleanly. Where Markov chains earn their place in the modern stack, where they were displaced, and why the displacement happened. What a Markov chain actually is A Markov chain is a sequence of states where the probability of moving to the next state depends only on the current state. That property — the Markov property — is the defining constraint. Formally, P(Xt+1 Xt, Xt-1, …, X0) = P(Xt+1 Xt). The mechanics are a transition matrix: a square table where entry (i, j) is the probability of moving from state i to state j. Multiply the current state distribution by the matrix, and you get the next-step distribution. Iterate, and you get a sequence. That’s the entire machinery. No hidden layers, no embeddings, no attention. Just states and probabilities. Why the memoryless constraint matters The constraint is what makes Markov chains tractable and what limits them. Tractable because the transition matrix is small relative to the data it summarises, and the math is closed-form. Limiting because most interesting sequences are not memoryless. Natural language depends on context that extends far beyond the previous token. Images depend on global structure, not local pixel adjacency. The Markov property is a strong assumption, and it breaks on exactly the workloads that modern generative AI cares about. Where Markov chains still pull weight Despite the displacement at the generative front-end, Markov chains remain load-bearing in several places — usually as components inside a larger system, not as the system itself. PageRank and graph-ranking algorithms PageRank models the web as a Markov chain over pages, with transition probabilities defined by hyperlinks. The stationary distribution of that chain is the ranking signal. Google’s original formulation worked over what Built In described as a corpus that grew to over 130 trillion indexed pages (published-survey; Urwin, Built In, 2024). PageRank is still one of the cleanest production examples of a Markov chain doing exactly what it was designed for: computing a stable distribution over a memoryless state graph. Hidden Markov Models in speech and bioinformatics HMMs — a chain of hidden states emitting observable signals — dominated speech recognition for decades and still appear in production hybrids alongside neural acoustic models. They also remain common in bioinformatics for gene-sequence annotation. The pattern: when the underlying state really is approximately memoryless given the previous state, the model fits the data. Sampling chains inside diffusion training Modern diffusion models use a Markovian forward process: noise is added to an image over T steps, where each step depends only on the previous one. The reverse process — what the trained model learns — is also Markovian by construction. So even though the architecture generating images is a U-Net, the training framework is a Markov chain. This is why “diffusion is the new Markov chain” is technically wrong but rhetorically close to something true. Reinforcement learning’s MDP formulation Markov decision processes underlie most RL algorithms. The agent’s state-transition model is assumed Markovian — a simplification that breaks for partially observable environments (handled separately as POMDPs). How does generative AI use Markov chains today? Inside the modern stack, Markov chains operate as structural scaffolding, not as generators. The headline generative architectures — transformers, diffusion models, GANs — are not Markov chains and were not built from them. Workload Generator architecture Where Markov property appears Text generation (chatbots, LLMs) Transformer with self-attention Not used in generation; tokens condition on full context Image generation (Stable Diffusion, SDXL) U-Net denoiser Forward/reverse noise process is Markovian Image generation (GAN-class) Generator + discriminator networks Not used; no sequential state model Speech recognition (legacy/hybrid) DNN acoustic model HMM still common for decoding Web ranking — PageRank Markov chain Game AI (NPC behaviour) Behaviour trees, RL policies Sometimes as state-transition controller The honest summary: if you’re shipping a chatbot, a diffusion image model, or a code-generation tool, you are not using a Markov chain as the generator. If you’re building a retrieval-ranking pipeline, decoding speech, or modelling an RL environment, you probably are — but not for generation. Why Markov chains lost the text-generation race Before transformers, statistical n-gram models — a direct generalisation of Markov chains, where the next token depends on the previous n tokens rather than just the previous one — were the dominant approach to language modelling. n-gram models are higher-order Markov chains. They worked, within sharp limits. The limits broke at two places. First, the curse of dimensionality: the transition table for a vocabulary of 50,000 tokens and a context window of even 5 grows to 50,0005 entries. Most of those entries have zero observed evidence, so the model needs aggressive smoothing and back-off heuristics that introduce their own errors. Second, long-range dependency: a sentence’s grammaticality often depends on agreement across 20–40 tokens, well beyond what any tractable n-gram model can capture. Transformers replaced this entire stack by conditioning on all prior tokens through attention, learning the dependency structure rather than assuming it. The Markov property was the assumption that no longer paid for itself. How does the choice differ for image generation? This is where the spoke connects to its parent: the architecture question for image generation today is GANs versus diffusion models, and Markov chains sit in a specific structural role inside the diffusion side. A GAN trains a generator network against a discriminator in an adversarial game. There is no state-transition sequence; the generator maps a noise vector directly to an output image in a single forward pass. Inference is fast. Training is unstable, mode collapse is a known failure mode, and controllability is limited. A diffusion model defines a Markov chain that gradually adds Gaussian noise to a training image across T steps (typically 1,000), then trains a neural network to reverse one step of that chain. Inference runs the reverse chain — historically 1,000 steps, now often 20–50 with samplers like DDIM or DPM-Solver. Training is stable, sample quality is currently the state of the art for general image synthesis, but inference is materially slower than a single GAN forward pass. The Markov-chain framework is what makes diffusion training tractable: the model only needs to learn a single denoising step, and the chain structure guarantees that iterating that step samples from the target distribution. Drop the Markov property and the math no longer closes. That is the precise, modern role of Markov chains in generative AI: a training-time framework for diffusion, not a generator in their own right. The parent article on GAN vs diffusion architecture trade-offs develops the broader selection criteria. Common misconceptions worth correcting A few patterns we see often when reviewing how teams describe their AI stack: “The chatbot uses a Markov chain to pick the next word.” It does not. Production LLMs use transformer decoders. The next-token distribution conditions on the entire context window through self-attention. “Markov chains power GANs.” They do not. GANs have no sequential state model. The confusion sometimes comes from conflating GANs with diffusion models. “Markov chains and neural networks are alternatives.” They occupy different layers. A neural network can parameterise the transition probabilities of a Markov chain, and frequently does in modern hybrid systems. “Markov chains can’t model long-range dependencies, so they’re obsolete.” The first half is true. The second is not — they are excellent at the workloads where the Markov property actually holds, and those workloads still exist. What this means for architecture decisions If a team is choosing a generative architecture, the Markov chain is rarely the right unit of analysis. The real decision is between transformer, diffusion, GAN, or a hybrid — and the trade-off space is training stability versus inference speed versus controllability versus dataset requirements. We cover that decision space in our work on the GenAI feasibility assessment and in the parent comparison of diffusion-model architecture. What Markov chains offer to the decision is more conceptual: they are the cleanest way to understand why diffusion models train stably (the Markov property makes the loss decompose across timesteps) and why simple sequential models fail at language (the property is too strong for the data). For teams shipping production systems, the practical answer is to treat Markov chains as a vocabulary item — useful for reading papers, debugging RL setups, or reasoning about diffusion schedules — rather than as a candidate architecture for a new generative product. FAQ What is the architectural difference between GANs and diffusion models? A GAN pairs a generator network with a discriminator in an adversarial training loop; inference is a single forward pass from noise to image. A diffusion model defines a Markov chain that adds noise over many steps and trains a network to reverse one step at a time; inference runs that reverse chain across tens to thousands of iterations. When does a GAN outperform a diffusion model for image generation — and when is it the other way around? GANs win on inference latency and compactness when single-pass generation matters (real-time avatars, on-device synthesis). Diffusion models win on sample quality, training stability, and controllability for general-purpose image and video generation. The parent article on diffusion architecture covers the trade-off in depth. Why are diffusion models slower at inference than GANs, and what does that cost in production? Diffusion sampling runs the reverse Markov chain over T steps. Even with accelerated samplers (DDIM, DPM-Solver) reducing T from 1,000 to 20–50, each step is a full network forward pass. A GAN does one. The cost shows up as latency per image and GPU-hours per batch. Which is more stable to train, GANs or diffusion models, and what failure modes does each introduce? Diffusion is materially more stable. GAN training is notoriously prone to mode collapse, training divergence, and discriminator-generator imbalance. Diffusion’s loss is a denoising objective with no adversary, which makes it well-behaved but expensive. How do controllability and conditioning flexibility compare between GANs and diffusion models? Diffusion models support richer conditioning through classifier-free guidance, ControlNet, and cross-attention with text encoders. GAN controllability is typically narrower and requires architecture-specific work. When does a hybrid approach (diffusion-GAN, distilled diffusion) earn its complexity? When you need diffusion-grade quality at GAN-grade latency. Distillation (e.g. consistency models, LCM) collapses the reverse chain into 1–4 steps and is the most common production hybrid. What does the choice between GAN and diffusion mean for required dataset size and compute? Diffusion models typically need larger and more diverse datasets to reach quality, and meaningfully more training compute. GANs can sometimes train on smaller datasets but are harder to stabilise. References Urwin, M. (2024, October 22). Markov Chain Explained. Built In. Wu, H., Mardt, A., Pasquali, L., & Noé, F. (2019). Deep Generative Markov State Models. arXiv:1805.07601. Zewe, A. (2023, November 09). Explained: Generative AI. MIT News.
