Hazards: When the Line Stalls

The dream and the catch

In the last two guides we built the picture: pipelining runs a laundry assembly line, so while one load dries another washes and a third folds. Our five-stage pipeline keeps five instructions in flight at once — fetch, decode, execute, memory, write-back — and the pipeline registers between stages carry each instruction's half-finished work forward one tick at a time. In the perfect case a new instruction enters every clock cycle, and the machine finishes one instruction per cycle even though each still takes five cycles end to end.

But the laundry analogy hid an assumption: that the loads are independent. Real instructions are not. One instruction often needs a result the instruction just ahead of it hasn't produced yet; a branch may change which instruction even comes next; and two instructions may both want the same piece of hardware in the same cycle. When the next instruction can't safely advance on schedule, we call it a hazard. A pipeline hazard is exactly a situation where letting the pipeline keep marching at full speed would give the wrong answer.

Structural hazards: two hands reaching for one tool

The simplest hazard is purely about plumbing. A structural hazard happens when two instructions in different stages need the same hardware resource in the same cycle, and there's only one of it. Imagine a laundry with a single machine that both washes and dries: the moment one load wants to wash while another wants to dry, somebody has to wait. The conflict isn't about data being wrong — it's about there not being enough metal to go around.

The classic example is memory. In a given cycle one instruction may be in the fetch stage reaching into instruction memory to grab the next opcode, while another instruction four stages ahead is in the memory stage reaching into data memory for a load or store. If those were the same single memory port, they'd collide every time the pipeline is full. This is exactly why a pipelined machine leans toward a Harvard-style split — separate instruction and data caches — so fetch and memory access can happen at once. Give each contender its own copy of the resource, and the structural hazard simply disappears.

Data hazards: the result isn't ready yet

The deeper trouble is dependence between values. A data hazard arises when one instruction needs to read a register that an earlier, still-in-flight instruction is going to write. The textbook case is a read-after-write dependence: the program means 'first compute x, then use x', but in a pipeline the second instruction reaches the stage where it reads x while the first instruction is still on its way to writing x. The order on paper is fine; the order in time inside the pipe is the problem.

Let's make it concrete with two RISC-V instructions, where the second adds using x1 — the very register the first just produced. Trace them through the five stages and watch where the danger sits.

  add  x1, x2, x3   # writes x1
  sub  x5, x1, x4   # reads  x1

cycle:        1    2    3    4    5    6
  add        IF   ID   EX  MEM   WB
  sub             IF   ID   EX  MEM   WB
                       ^
          sub reads x1 in ID (cycle 3),
          but add writes x1 in WB (cycle 5)
          --> sub would read a STALE x1

The sub reads x1 two cycles before the add finishes writing it — a read-after-write data hazard.

There are two honest ways out, and the next guide is devoted to them, so here we only name the shapes. The cheap fix is forwarding (also called bypassing): the value of x1 actually exists at the ALU's output in cycle 3 — it just hasn't been written to the register file yet — so we add wires that route it straight from where it was computed to where it's needed, no waiting. The fallback is a stall: when even forwarding can't deliver the value in time, the pipeline freezes the dependent instruction for a cycle or two and lets the producer catch up.

The load-use hazard: even forwarding can't beat physics

One data hazard refuses to be fully forwarded, and it's worth singling out. When the producer is a load — an instruction that fetches a value from memory — the value doesn't exist until the very end of the memory stage. If the next instruction needs that value in its execute stage, there is simply no earlier moment to forward it from: the data is born one cycle too late. This is the load-use hazard, and unlike an add-to-add dependence, it forces at least a one-cycle bubble even with full forwarding in place.

A bubble is just a no-op pushed into the pipeline — a cycle where the dependent instruction marks time and nothing useful retires. Hardware can detect a load-use hazard and insert the bubble automatically, or a clever compiler can hide it by scheduling some unrelated, independent instruction into that slot so the cycle isn't wasted. Either way, the cost is real, which is why this small case looms large in performance tuning: tight loops full of pointer-chasing loads are exactly where these bubbles pile up.

Control hazards: which way does the road fork?

The third family is about what comes next at all. A control hazard (or branch hazard) appears at a conditional branch: the machine fetches instructions in a straight line every cycle, but a branch may redirect the flow — and the pipeline doesn't learn whether the branch is taken, or where it goes, until several stages in. Meanwhile it has already fetched the instructions that physically follow the branch. If the branch turns out to jump elsewhere, those already-fetched instructions are wrong and must be discarded.

Throwing them out is a pipeline flush: the wrongly-fetched instructions are turned into bubbles before they can change any register or memory, so the program's visible result stays correct — you only paid in wasted cycles, never in wrong answers. The number of cycles lost per misguess is how deep into the pipe the branch's outcome was resolved. Resolve it in stage 3 and you flush two instructions; resolve it later and you flush more.

Rather than always stall and wait, real machines place a bet. A branch predictor is exactly that — a guess about which fork in the road the program will take — and the pipeline speculatively fetches down the predicted path. Guess right and you paid nothing; guess wrong and you flush and restart. This is the seed of the next guide, and also a quiet warning sign: this very habit of running ahead on a guess is what the Spectre and Meltdown vulnerabilities later learned to exploit. For now, just hold the shape: predict, run ahead, and flush on a miss.

Adding it up: what hazards do to CPI

Tie it back to the iron law: run time = instruction count x CPI x cycle time. Pipelining's whole promise is to drive CPI down toward the ideal of 1. Every stall and every flush adds idle cycles that don't retire an instruction, so they raise the effective CPI above 1. If, say, 20 percent of instructions are loads and a fifth of those trigger a one-cycle load-use bubble, and 15 percent are branches that mispredict a tenth of the time costing two cycles each, you can literally sum those penalties: effective CPI is roughly 1 plus all the bubble cycles per instruction.

This frames the honest tradeoff that closes the rung. A deeper pipeline chops work into more, smaller stages, so each stage is shorter and the machine can be clocked faster — fewer nanoseconds per cycle. But more stages mean the branch outcome is resolved later, so each misprediction flushes more instructions, and each stall is comparatively costlier. Deeper pipes clock faster yet pay more per stall and per misguess; the sweet spot is a balance, not a maximum, and chasing clock rate alone is the megahertz myth all over again.