The Decoder-Only Blueprint

One shape won

The original 2017 Transformer had two halves: an encoder that read a whole input at once, and a decoder that wrote an output one token at a time. Modern general-purpose LLMs threw the encoder away. A decoder-only architecture keeps only the writing half and uses it for everything — reading your prompt is just writing tokens it already knows. GPT, Llama, Mistral, Qwen, DeepSeek: under the branding, they are all the same shape.

Why did this minimalist design win? Because it makes one job — next-token prediction — and does it at every position simultaneously during training. Each token may only look backward, enforced by causal masking, so a single forward pass over a sentence yields a prediction target at every position. That training efficiency, more than any clever module, is the quiet reason the decoder-only stack ate the field.

The stack is boringly repetitive

A modern LLM is a stack of identical blocks — 32 of them in an 8B model, 80+ in the largest. Tokens enter as embeddings, flow up through every block, and exit as logits over the vocabulary. Each block does exactly two things: it mixes information between tokens (attention) and then transforms each token on its own (a feed-forward network). Nothing else. The depth, not the cleverness of any one layer, is where the capability lives.

The residual stream is the highway

Picture a wide pipe running straight up through every block — the residual stream. A block never overwrites it; it reads from the pipe, computes a small update, and adds that update back. Attention adds its contribution, the feed-forward layer adds its own, and the original signal keeps flowing untouched alongside. This `x = x + sublayer(x)` pattern is why a 100-layer model can train at all: gradients have an unbroken path back to the input.

x_{\ell+1} = x_{\ell} + F_{\ell}\!\left(\mathrm{norm}(x_{\ell})\right)

Each block reads the residual stream, computes a small update on a normalized branch, and adds it back — never overwriting.

for block in blocks:           # the entire model, conceptually
    x = x + attention(norm(x))  # mix across tokens, add to the stream
    x = x + feedforward(norm(x))# transform each token, add to the stream
logits = unembed(norm(x))       # final read-out over the vocabulary

The decoder-only loop. Note that `norm` is applied to the branch, not the stream — that is pre-norm.

Normalize before, not after

Where you put the normalization layer turns out to matter enormously. The 2017 design used post-norm (normalize after adding to the stream). Every serious modern model uses pre-norm: normalize the input to each sub-layer, leave the residual highway untouched. Normalization placement is the difference between a deep model that trains smoothly and one whose loss explodes in the first thousand steps.

And the normalizer itself got simpler. Classic LayerNorm both re-centers (subtract the mean) and re-scales. Modern stacks use RMSNorm, which skips the mean-subtraction entirely and only divides by the root-mean-square. It is cheaper, and empirically the recentering was doing almost nothing. We will dissect RMSNorm and the rest of the block in the next guide.

Tying the ends together

Two layers bracket the whole stack: an embedding table that turns token IDs into vectors at the bottom, and an unembedding matrix that turns the final vector into vocabulary scores at the top. Weight tying often makes these the same matrix — the input and output embeddings are shared — which saves a chunk of parameters in smaller models and ties the geometry of 'meaning in' and 'meaning out' together.