encoder-decoder
/ en-KOH-dur dee-KOH-dur /
An encoder-decoder is a two-part design for tasks where the input and the output are different kinds of things — translate English into French, turn a long article into a short summary, describe a photo in words. The trouble is that the input and output don't line up one-to-one: a five-word English sentence might need seven French words, in a different order. The encoder-decoder solves this by splitting the work in two: an encoder reads and digests the entire input into a compact internal meaning, and a decoder then generates the output from that meaning, one piece at a time.
Think of a skilled human interpreter. They don't translate word by word as you speak; they listen to your whole sentence first, grasp what you meant — a thought now held in their head, free of the original words — and then express that thought naturally in the other language. The encoder is the listening-and-understanding half; the meaning it forms is a bundle of numbers sometimes called the context. The decoder is the speaking half, building the output step by step, each new piece guided both by that context and by what it has already produced.
This pattern is everywhere: machine translation, summarization, speech-to-text, image captioning, even drawing a clean image from a noisy one. Early versions had a real bottleneck — the encoder had to cram a long input into one fixed-size context, and long inputs got garbled. The fix, called attention, lets the decoder look back at all the encoder's intermediate notes and focus on the relevant ones as it writes each word. Attention proved so powerful that it grew into the Transformer, which is itself built on the encoder-decoder idea — so this old design quietly underpins much of modern AI.
Translating "the cat sat": the encoder reads all three words and folds them into a context vector capturing the meaning; the decoder then emits "le," then "chat," then "s'est assis" — each French word chosen using the context and the words already produced.
Understand the whole input first, then speak the output piece by piece.
The encoder's job is to understand; the decoder's job is to generate. Cramming a long input into a single fixed context was the original weak point — attention, which lets the decoder revisit the whole input, was the breakthrough that fixed it and led to the Transformer.