arithmetic intensity
Picture a delivery driver paid only for parcels delivered, but who must drive to a distant warehouse to pick up each load. If every warehouse run yields one tiny parcel, the driver spends all day on the road and delivers almost nothing — the trips dominate. If each run fills the van with hundreds of parcels, the driving is amortised and the driver is productive. Arithmetic intensity asks the same question of a computation: for every byte you haul up from memory, how many useful arithmetic operations do you get out of it?
Formally, arithmetic intensity is the ratio of floating-point operations performed to bytes of data moved to and from main memory, often written I = flops / bytes. A vector add c = a + b reads 16 bytes (two doubles) and writes 8 to produce 1 flop, so its intensity is about 1/24 flop per byte — very low; it is starved for arithmetic, limited entirely by how fast memory can feed it (memory-bound). A dense matrix multiply, by contrast, does 2*n^3 flops while moving only on the order of n^2 data (with good blocking), so its intensity grows like n — very high; it can keep the arithmetic units busy and is limited by raw compute speed (compute-bound). The dividing line, the 'ridge point', is where the machine's peak flop rate equals its peak memory bandwidth times the intensity.
Arithmetic intensity is the one number that tells you, before you optimise anything, whether you are fighting the arithmetic units or the memory system — and therefore which optimisations can possibly help. If a kernel is memory-bound (low intensity), buying a faster processor or hand-tuning the inner loop does nothing; you must instead raise the intensity by reusing data (blocking, fusing operations) or simply move less of it. If it is compute-bound (high intensity), then vectorization and faster arithmetic pay off. Plotted against the roofline model, intensity is the horizontal axis that places a kernel under either the bandwidth ceiling or the compute ceiling.
Three kernels by intensity: a vector add (~0.04 flop/byte, deeply memory-bound), a matrix-vector product A*x (~0.25 flop/byte, still memory-bound), and a blocked matrix-matrix multiply (intensity grows with block size, can exceed 10 flop/byte and become compute-bound). Only the last can approach the machine's peak flop rate; the first two are capped by memory bandwidth no matter how clever the arithmetic.
Intensity = flops per byte moved; it decides whether memory or arithmetic is your wall.
Intensity depends on what counts as 'data moved' — for the same algorithm it is much higher if data stays in cache than if it streams from DRAM. So a kernel can be memory-bound on a big problem and compute-bound on a small one that fits in cache; the number is not a fixed property of the math alone.