pipelining
Imagine a laundromat with three machines: a washer, a dryer, and a folding table. You have four loads of laundry. The slow way is to wash load 1, dry load 1, fold load 1, and only THEN start washing load 2. Each load takes three steps, so four loads take twelve steps' worth of time, and most of the time two of your three machines sit idle. The clever way is an assembly line: while load 1 is drying, start washing load 2; while load 1 is being folded and load 2 is drying, start washing load 3. Now all three machines run at once. This is pipelining.
A processor pipelines instructions the same way. Executing one instruction takes several steps — fetch it from memory, decode what it asks for, do the arithmetic, touch memory, write the answer back. A non-pipelined CPU finishes one instruction completely before starting the next. A pipelined CPU breaks the work into stages and overlaps them: while one instruction is in the execute stage, the next is being decoded, and the one after that is being fetched. With k stages, up to k instructions are in flight at the same moment, each in a different stage. The clock ticks once per stage, and on every tick each instruction advances one stage and a finished instruction pops out the end.
Pipelining is the first great performance idea in computer architecture, and it is nearly free: you reuse the same hardware, just adding registers between the stages to hold each instruction's state. But it has a famous catch that beginners must internalize: pipelining raises throughput (how many instructions finish per second) without lowering the latency of any single instruction. One load of laundry still takes wash-plus-dry-plus-fold; you have not made any single instruction faster, you have only stopped the machines from idling. The rest of this field is about the hazards that break the smooth overlap and the tricks that keep the line moving.
Five instructions, five-stage pipeline. Cycle 1: I1 fetch. Cycle 2: I1 decode, I2 fetch. Cycle 3: I1 execute, I2 decode, I3 fetch. By cycle 5 the pipe is full: I1 write-back, I2 memory, I3 execute, I4 decode, I5 fetch — five instructions in flight at once. From cycle 5 onward, one instruction finishes every single cycle.
Once the pipeline is full, ideal throughput is one instruction per cycle even though each instruction still took five cycles end to end.
The single most common misconception: pipelining does not make any individual instruction faster. It overlaps instructions to raise throughput; the latency of one instruction is unchanged (or slightly worse, because of the added pipeline registers).