the roofline model
/ ROOF-line /
You bought a blazing-fast sports car, but you keep driving it through a town with a single narrow bridge that everyone must cross. Past a point, a faster engine buys you nothing — the bridge, not the engine, decides how fast traffic flows. The roofline model is a simple, honest picture that answers the same question for a computer: is my program limited by how fast the chip can do arithmetic, or by how fast it can move data from memory? It draws both limits on one graph so you can see which one is your bridge.
Here is the graph. The horizontal axis is arithmetic intensity — how many arithmetic operations the program does per byte of data it moves from memory (operations per byte). The vertical axis is achievable performance (operations per second). Two ceilings form a roof shape. A slanted ceiling rising from the left is the memory limit: performance equals memory bandwidth times arithmetic intensity, so at low intensity you are bandwidth-bound and going wider in arithmetic does not help. A flat ceiling on the right is the compute limit: the machine's peak arithmetic rate, which you cannot exceed no matter how much data you reuse. Where the slant meets the flat is the 'ridge point' — the intensity you must reach to stop being starved by memory.
Why this matters, honestly. A program's arithmetic intensity tells you which regime you are in. Low intensity (a few operations per byte, like adding two big arrays, which reads 8 bytes to do 1 add) sits under the slanted roof — you are memory-bound, and a faster arithmetic unit or a wider SIMD does nothing; only more bandwidth or better data reuse helps. High intensity (many operations per byte, like a well-blocked matrix multiply that reuses each loaded value many times) reaches the flat roof — now you are compute-bound and faster arithmetic finally pays. The roofline keeps beginners honest: it explains why a GPU's enormous peak arithmetic rate is wasted on memory-bound kernels, and why the famous fix is to raise arithmetic intensity (reuse data in fast on-chip memory) rather than to buy a bigger arithmetic unit.
Adding two arrays, c[i]=a[i]+b[i]: it reads 8 bytes (two floats) and writes 4, doing just 1 add — arithmetic intensity about 1 operation per 12 bytes, very low, so it lands under the slanted memory roof and is bandwidth-bound. A blocked matrix multiply reuses each loaded value across many multiply-adds, reaching high intensity and the flat compute roof — only then does raw arithmetic speed matter.
The roof's slanted part is the memory-bandwidth limit; the flat part is the peak-compute limit. Arithmetic intensity decides which one caps you.
The roofline is a ceiling, not a promise: real code can fall well below either limit for many reasons. Its value is diagnostic — it tells you whether to chase more bandwidth/data-reuse (memory-bound) or more arithmetic throughput (compute-bound), so you stop optimizing the wrong thing.