The three LLM types are encoder-only, decoder-only and encoder-decoder, and the split between them comes down to one question: which tokens each token is allowed to attend to. In an encoder, every token attends to every other token, so it reads the whole input at once. In a decoder, self-attention is causally masked, so a token sees only what comes before it, which is what lets the model generate text one token at a time. An encoder-decoder model joins the two with cross-attention, where the decoder looks back at the encoder’s output while it writes.
Encoder vs decoder in a transformer: what each token can see
Both halves of a transformer are built from the same parts, alternating attention and feed-forward layers sitting between an embedding layer and an un-embedding layer. Sebastian Raschka puts the practical difference well: both use the same self-attention layers, but encoders are designed to learn embeddings for predictive tasks such as classification, while decoders are designed to generate new text.
Mechanically, the difference is the mask. An encoder processes the entire input in one go, so there is no need for causal masking. A decoder cannot attend to tokens that have not been generated yet, so its self-attention is masked to keep information from flowing backwards from future positions, which preserves the autoregressive property.
Counting sublayers makes the three types concrete. In the original design each encoder layer has two sublayers (self-attention and a feed-forward network) and each decoder layer has three (causally masked self-attention, cross-attention and a feed-forward network). A “decoder-only” model is not literally the original decoder: with no encoder, cross-attention has nothing to attend to, so its layers drop back to two sublayers, causally masked self-attention plus feed-forward.
The three LLM types side by side
| Encoder-only | Decoder-only | Encoder-decoder | |
|---|---|---|---|
| Self-attention | All-to-all (bidirectional) | Causally masked, left to right | Bidirectional in the encoder, causal in the decoder |
| Cross-attention | None | None | Decoder attends to encoder output |
| Sublayers per layer | 2 | 2 | 2 in the encoder, 3 in the decoder |
| Typical pretraining | Masked language modelling | Next-token (causal) language modelling | Corrupted-span reconstruction, for example T5’s span replacement |
| Suited to | Classification, named entity recognition, extractive question answering, embeddings and retrieval | Text generation, dialogue, open-ended Q&A | Summarisation, translation, generative question answering |
| Well-known examples | BERT, ModernBERT | GPT series, LLaMA, Chinchilla, BLOOM | T5, BART, Flan-T5 |
Inside an encoder-decoder model: how cross-attention links input to output
An encoder-decoder model reads the input with one stack and writes the output with another, connected by cross-attention. The encoder turns the input tokens into contextualised representations, each token having “mixed” in information from the rest of the input. The decoder then generates the output autoregressively, and in each of its layers the queries come from the decoder while the keys and values come from the encoder’s output, so every output position can look at every input position.
Because the encoder’s output is computed before decoding starts, cross-attention needs no mask; only the decoder’s own self-attention is causal. This separation lets input and output differ in length and structure.
The original transformer used this encoder-decoder design. The 2017 “Attention Is All You Need” paper used a six-layer encoder and a six-layer decoder for machine translation. Its big model reached 28.4 BLEU on WMT 2014 English-to-German, more than 2.0 BLEU above the best previously reported results, after 3.5 days of training on 8 P100 GPUs. Instruction following uses the same structure: in T5 the encoder encodes the instruction and the decoder autoregressively generates the reply.
BART is a useful reference point because it combines bidirectional and autoregressive transformers. Its authors describe it as generalising BERT, through its bidirectional encoder, and GPT, through its left-to-right decoder.
Why are most large language models decoder-only?
Decoder-only is the default for modern LLMs: the Hugging Face LLM course says most of them use it, and the Zhao et al. survey notes that only a small number of LLMs are built on the encoder-decoder architecture. Why that happened is less settled than the headcount suggests, so we treat the dominance as a judgement about practice rather than a proven architectural verdict.
Wang et al. (2022) found that causal decoder-only models trained on autoregressive language modelling showed the strongest zero-shot generalisation after purely unsupervised pretraining. GPT-3 then showed what a large causal decoder could do with in-context learning, completing a prompt’s word sequence without any gradient update. Scaling laws have also been widely observed in causal decoders, which made scaling up an important strategy for increasing their capacity. The Ettin authors add a plainer reason: the community focuses on decoders because they are easier to use for text generation.
The case against treating it as settled is just as well sourced. The same Zhao et al. survey states that no formal study has shown theoretically that decoder-only has an advantage over other architectures, and that encoder-decoder models at large scale have been studied less. Wang et al. found that non-causal models trained with masked language modelling and then multitask fine-tuned performed best of all in their experiments. Google’s T5 work found encoder-decoder models generally outperformed decoder-only language models. The study “Encoder-Decoder or Decoder-Only? Revisiting Encoder-Decoder Large Language Model”, covering about 150M to 8B parameters, found decoder-only models more compute-optimal during pretraining, but encoder-decoder models scaled comparably and, after instruction tuning, matched or beat them on downstream tasks with substantially better inference efficiency.
Decoder-only wins on zero-shot use straight after pretraining; non-causal and encoder-decoder models catch up or pull ahead after multitask fine-tuning or instruction tuning.
When is an encoder-only model like BERT a better fit than a decoder-only model like GPT?
Pick an encoder-only model when the job is to understand or represent text rather than to write it: classification, named entity recognition, extractive question answering, retrieval and embeddings. BERT was built to pretrain deep bidirectional representations by conditioning on both left and right context in every layer, and it can be fine-tuned with a single extra output layer for tasks such as question answering and language inference.
The flip side is a hard limit. Because it has no decoder, BERT cannot easily be prompted and cannot generate high-quality text.
Encoders also compare well on size. The ModernBERT authors describe encoder-only models as offering a strong performance-size trade-off for retrieval and classification compared with larger decoder-only models, and ModernBERT brings a native 8192-token sequence length to the type. The Ettin suite trained paired encoders and decoders on the same recipe: encoders won at classification and retrieval, decoders won at generation, and a 400M encoder beat a 1B decoder on MNLI. The same study found that continued training to adapt a decoder for encoder tasks, or the reverse, did worse than using a model trained on the matching objective from the start. A language-specific example points the same way: Slovenian BERT did better on classification while Slovenian T5 looked stronger on generation.
For a classifier or retrieval component in production, we would start from a strong encoder before reaching for a generative LLM, because the encoder is built for the task. The wider trade-off between model sizes is covered in our piece on small vs large language models.
Encoder-decoder vs decoder-only for translation and summarisation
For tasks where the output depends closely on the input, such as translation and summarisation, encoder-decoder is the natural fit, and in translation, single-stack language models can match it when their mask lets them see the whole source. The Hugging Face course lists summarisation, translation and generative question answering as what sequence-to-sequence models suit best.
The mechanism explains why. A Hugging Face-hosted paper describes an “attention degeneration” problem in decoder-only models applied to sequence-to-sequence tasks: as generation goes on, less and less attention is paid to the source sequence. An encoder-decoder model keeps the source in a separate representation that every decoding step can reach through cross-attention.
The gap is not fixed, though. An ICML 2022 study of machine translation found that causal masking and a language-modelling objective on the source sequence hurt translation quality, but that decoder-style language models with full-visible masking over the source performed on par with encoder-decoder models on supervised bilingual and multilingual translation. The same study found architectural differences mattered most at small scale and narrowed as parameter count grew. A 2024 study of decoder-only and encoder-decoder models for Indian-language translation described encoder-decoder models as reliable, and decoder-only models as producing fluent translations with lower computational complexity.
BART reported new state-of-the-art results on abstractive dialogue, question answering and summarisation tasks, with gains of up to 6 ROUGE. In 2025 Google’s T5Gemma, which adapts decoder-only Gemma 2 weights into an encoder-decoder, argued that encoder-decoder models often excel at summarisation, translation and QA thanks to inference efficiency and a richer encoder representation of the input.
The decision depends on what else the model must do. If one model has to handle translation alongside open chat, a strong decoder-only model with a well-built prompt is the simpler deployment; if the system does nothing but transform documents, an encoder-decoder deserves a proper evaluation rather than being ruled out by default.
Pretraining objectives are a separate choice from architecture
Masked, autoregressive and prefix language modelling are training tasks, and they are independent of any particular architecture even though they are usually discussed alongside transformers. In practice they line up with the types: causal language modelling for decoder-only models, masked language modelling for encoder-only models, and masked span prediction for encoder-decoder models such as CodeT5.
The pairing is not fixed by the architecture. A prefix language model is a decoder-only architecture whose mask lets a prefix be read bidirectionally while the rest is generated causally, which makes it resemble an encoder-decoder. GLM-130B and U-PaLM are examples of prefix decoders. The 2025 scaling study trained its encoder-decoder with prefix language modelling and its decoder-only model with causal language modelling, and Wang et al. tested three architectures with two pretraining objectives. When a comparison of two LLM types reports a winner, it is worth checking whether the architecture or the objective made the difference.
How the three types differ when you serve them
An encoder-only model handles its input in one forward pass and produces a label or vector, whereas a decoder generates output one token at a time, each step conditioned on everything generated so far.
On encoder-decoder against decoder-only, two published results favour encoder-decoder. Google reported that T5Gemma 9B-2B delivered a significant accuracy gain over the 2B-2B model with latency nearly identical to Gemma 2 2B. The 2025 scaling comparison likewise reported substantially better inference efficiency for its encoder-decoder models after instruction tuning. Both results come from particular model pairs and benchmarks, not a general cost model. For decoder-only serving, our article on LLM quantization and where accuracy breaks goes into the details. TechnoLynx’s generative AI work is described on its generative AI page.
LLM types are only one corner of generative AI
All three types are transformer language models, but the same building blocks reach beyond text: DALL-E pairs a decoder-only transformer with a variational autoencoder, Parti is an encoder-decoder that writes image tokens, and Muse is an encoder-only model that predicts masked image tokens. Generative models beyond LLMs, such as GANs, diffusion models and VAEs, are covered in what types of generative AI models exist beyond LLMs and when each applies. For the basics of how a language model predicts text in the first place, see Understanding Language Models: How They Work.
Encoder-decoder models at large scale remain less investigated than causal decoders. The Zhao et al. survey calls for more varied exploration of architecture design, and the 2025 results suggest that exploration could pay off. Until it does, the honest rule is to treat decoder-only as the convenient default and to test the other two types wherever the task is classification, retrieval or input-bound transformation.
