JOVANA
Explore Library Glossary Getting Started Three Levels Fields How it works Mission
Join the mission
All guides

Forwarding and Stalls

The last guide showed where the assembly line jams. This one builds the cures: forwarding, which hands a result straight to the instruction that needs it, and stalls, which insert a bubble when even forwarding cannot save you in time. We trace both on real instructions and count the cost in CPI.

Where we left the line jammed

In the last guide you met the data hazard: when one instruction needs a value that an earlier, still-in-flight instruction has not yet written back. The clean example is a read-after-write pair — an add that produces a register, immediately followed by a subtract that reads it. In the five-stage pipeline the add computes its answer in the EX stage but only stores it into the register file three cycles later in WB. The subtract, riding one stage behind, reaches its own register-read in ID before that write happens, and would read the stale old value. The naive fix was to freeze the line until the write lands — correct, but slow.

But look closely and something nags. The add already knew its answer at the end of EX, two whole cycles before it bothered to write it back. The subtract does not need that answer until its own EX, the very next cycle. The value exists; it is simply sitting in a pipeline register instead of in the register file where the subtract is looking. The whole problem is a delivery problem, not a computation problem — and that reframing is the key that unlocks the cheap cure.

Forwarding: hand the result straight across

Forwarding — also called bypassing — adds short wires that carry a freshly computed result backward from a later stage straight into the input of an earlier one, skipping the long round-trip through the register file. Picture the laundry line again: the folder needs a clean shirt, and instead of waiting for it to be carried all the way to the wardrobe and fetched back, the dryer simply hands the warm shirt directly across the bench. The shirt was ready; we just routed it the short way.

Concretely, the ALU result computed in the EX stage is latched into the EX/MEM pipeline register at the tick. A forwarding path runs from that register back to the ALU's input multiplexers. So when the subtract enters EX one cycle later, a small forwarding unit notices that one of its source registers matches the destination of the instruction now in MEM, and steers the multiplexer to grab the value off the bypass wire instead of the stale register-file read. There is even a second path from the MEM/WB register, to cover a result that is two instructions ahead. With both paths in place, an ordinary back-to-back arithmetic dependence costs zero extra cycles — the line never stalls at all.

Without forwarding (stall 2 cycles so WB precedes ID):
  add x1,x2,x3   IF ID EX ME WB
  sub x4,x1,x5      IF ID -- -- EX ME WB     <- two bubbles waiting

With forwarding (EX result bypassed into next EX):
  add x1,x2,x3   IF ID EX ME WB
  sub x4,x1,x5      IF ID EX ME WB           <- 0 bubbles
                          ^ x1 forwarded from EX/MEM into the ALU input
An arithmetic-to-arithmetic dependence: stalling costs two bubbles; forwarding the EX result costs none.

The load-use hazard: when even forwarding is one cycle too late

Forwarding feels almost magical, so it is tempting to believe it erases every data hazard. It does not, and the honest exception is worth understanding. A load does not produce its value in EX — in EX it only computes the memory address. The loaded data does not actually arrive until the end of the MEM stage, one stage later than an ALU result. If the very next instruction wants that loaded value, it needs it at the start of its own EX — which is the same cycle the load is still busy in MEM. The value simply is not born yet. No wire, however short, can forward a result backward in time.

This is the famous load-use hazard, and it costs exactly one unavoidable bubble. The hardware inserts a single stall: a hazard detection unit spots a load whose destination matches the next instruction's source, and freezes the IF and ID stages for one cycle while injecting a bubble — a do-nothing instruction whose control signals are all switched off — into EX. That one wasted cycle lets the load finish MEM; now its data sits in the MEM/WB register and can be forwarded into the dependent instruction's EX. So a load-use pair pays one bubble even with full forwarding, where an ordinary arithmetic pair paid none.

Counting the cost honestly in CPI

Every bubble is a wasted clock cycle, so hazards show up directly in the effective CPI — the real, measured cycles per instruction once stalls are counted. An ideal pipeline retires one instruction per cycle, an effective CPI of 1.0. Each stall adds a fraction. Suppose 20% of instructions are loads, and 40% of those are immediately followed by an instruction that uses the result. Then the load-use penalty adds 0.20 x 0.40 x 1 = 0.08 to CPI, lifting it from 1.0 to 1.08. Tiny here — but it scales with how dependence-heavy and how branch-heavy your program is.

Hold this against the iron law of performance: run time = instruction count x CPI x cycle time. Forwarding is wonderful precisely because it keeps CPI near 1 without lengthening the cycle time — it is a few short wires and one multiplexer choice, comfortably inside one stage. Stalls are the honest tax we pay only when physics forbids forwarding, as with the load-use case. The art of a good pipeline is to forward everything that can be forwarded so that the rare, genuinely unavoidable bubble is all that remains.

Putting it together: trace, detect, decide

It helps to see forwarding and stalling as one decision the hardware makes every cycle for the instruction entering EX. It asks: does either of my source registers match a result that is in flight ahead of me? If that result is already computed (an ALU op in MEM or WB), forward it — no cost. If it is a load that is exactly one step ahead and not yet out of MEM, there is no value to forward yet, so stall one cycle and then forward. Everything else flows untouched.

  1. An instruction reaches ID; the hazard unit reads its two source register numbers.
  2. Compare each source against the destination registers of the instructions now in EX/MEM and MEM/WB.
  3. If a match is an already-computed ALU result, set the forwarding multiplexer to pull it off the bypass wire — zero stall.
  4. If a match is a load still in MEM (the load-use case), freeze IF and ID, inject one bubble into EX, then forward next cycle.
  5. Otherwise let the instruction proceed normally, reading its operands from the register file.

Two cures, one philosophy: prefer the cheap fix and pay the expensive one only when forced. Forwarding handles the common, free case; the stall handles the rare case physics demands. The third kind of hazard — the control hazard of a branch, where we do not even know which instruction comes next — needs a different and more interesting cure, and that is exactly where the final guide of this rung picks up.