the limits of ILP

Imagine throwing more and more cooks into one kitchen to finish a single meal faster. At first it helps a lot, but soon the cooks are tripping over each other, the recipe's steps still have to happen in order, and the tenth cook adds almost nothing. The limits of ILP describe this diminishing return for a single instruction stream: beyond a certain point, making one core wider and cleverer extracts only a little more parallelism, at rapidly rising cost.

Several hard walls combine. True data dependences form chains that cannot be parallelized — if each step needs the previous result, no amount of hardware helps. Branch prediction, however good, eventually mispredicts and flushes speculative work; control dependences cap the useful look-ahead window. Memory latency and ambiguous load/store ordering stall things unpredictably. And the machinery to look further ahead — bigger reorder buffers, more reservation stations, wider issue, more renaming registers, the wakeup/select logic — grows in complexity roughly quadratically, so doubling the window for a few percent more performance is a terrible trade. Studies of 'ideal' ILP found that even with unlimited hardware, typical integer programs sustain only a handful of independent instructions per cycle.

This is one of the most consequential facts in modern computing. Around the mid-2000s, single-core ILP gains had flattened just as Dennard scaling ended and the power wall hit, so pouring transistors into a wider single core stopped paying off in either performance or energy. That is precisely why the industry pivoted to multicore — exploiting thread-level parallelism across many simpler cores — and later to specialized accelerators. The limits of ILP are not a temporary engineering gap; they are a structural ceiling rooted in the dependence structure of ordinary programs.