the roofline model
Before you spend a week optimising a kernel, you want a one-glance picture that says 'this code is as fast as this machine will ever let it go' or 'there is 10x left on the table, and here is what is holding you back'. The roofline model is exactly that picture: a single plot that draws the speed limits of a machine and lets you drop any kernel onto it to see, at a glance, whether memory bandwidth or raw compute is the wall, and how far from the wall you currently are.
The plot has arithmetic intensity (flops per byte moved) on the horizontal axis and attainable performance (flops per second) on the vertical, both on log scales. It shows two ceilings. A slanted line on the left is the bandwidth roof: performance = intensity * peak_bandwidth, because a low-intensity kernel can run no faster than memory can feed it. A flat line on the right is the compute roof: the machine's peak flop rate, which no kernel can exceed however much data it reuses. The two meet at the ridge point, intensity = peak_flops / peak_bandwidth. A kernel to the left of the ridge is memory-bound and pinned under the slanted roof; to the right it is compute-bound and pinned under the flat roof. Where your measured kernel sits versus its roof tells you both your ceiling and your headroom.
The model's value is that it turns 'how do I make this faster?' into two concrete questions. If your kernel is under the bandwidth roof, raising its arithmetic intensity (blocking, fusing loops, better data layout) slides it rightward toward the higher ceiling — buying a faster CPU would not help. If it is under the compute roof but well below it, the fix is vectorization, using fused multiply-add, or better parallelism. And if a kernel already touches its roof, you are done — no amount of cleverness beats the hardware, and chasing further is wasted effort. The roofline is honest about limits: it tells you not just how to go faster but when to stop.
On a machine with 1 Tflop/s peak compute and 100 GB/s bandwidth, the ridge point is at intensity 10 flop/byte. A vector add at 0.04 flop/byte is far left, capped at 0.04 * 100 = 4 Gflop/s — just 0.4 percent of peak, and no smarter arithmetic helps. A well-blocked matrix multiply at intensity 20 is right of the ridge and can approach the full 1 Tflop/s.
Slanted bandwidth roof, flat compute roof, meeting at the ridge — your kernel sits under one of them.
The basic roofline gives an optimistic upper bound: it assumes perfect overlap of compute and memory and full use of caches. Real kernels usually sit below the roof for reasons it does not show (latency stalls, poor vectorization, load imbalance), so 'below the roof' diagnoses headroom but not its exact cause.