cycles per instruction (CPI)

Imagine a factory line where the clock ticks once a second and each tick lets every workstation take one small step. Some products are simple and finish in one tick; others are fiddly and need four or five ticks each. If you want to know how long the day's run will take, you cannot just count the products — you need to know, on average, how many ticks each product eats. That average is exactly what cycles per instruction measures: on average, how many clock cycles does the processor spend per instruction?

CPI is the middle term of the iron law. CPU time = instruction count times CPI times clock-cycle time. It is computed as total clock cycles divided by total instructions executed. A simple non-pipelined design might take many cycles per instruction; an ideal pipeline aims for CPI near 1 (one instruction finishing every cycle); a superscalar machine that finishes several instructions per cycle can have a CPI below 1, which people often flip over and quote as IPC, instructions per cycle. Different instruction types contribute differently: if 25 percent of instructions are loads that take 5 cycles and the rest take 1, the average CPI is 0.25 times 5 plus 0.75 times 1, which is 2.

CPI is where the microarchitecture earns its keep, and it is honest precisely because it separates concerns. The instruction set and compiler mostly fix the instruction count; the clock-cycle time is mostly set by the technology and circuit design; CPI is where pipelining, caches, branch prediction, and out-of-order execution all show up. A cache miss inflates CPI by adding stall cycles; good branch prediction lowers it. This is why two chips with the same clock rate and running the same program can differ wildly in speed: they differ in CPI, and CPI is the most architecturally interesting of the three iron-law factors.