vectorization (SIMD)

Suppose you must add two columns of eight numbers each. You could write eight separate additions, one after another — or you could imagine a special calculator that, with one press, adds all eight pairs at once. Modern processors have exactly that calculator built in. SIMD, short for Single Instruction Multiple Data, is hardware that applies one operation to a whole little vector of values simultaneously, and vectorization is the act of rewriting (or letting the compiler rewrite) your loops to use it.

Concretely, a CPU has wide registers — say 256 or 512 bits — that hold not one number but a packed lane of several: a 512-bit register holds eight double-precision numbers or sixteen single-precision ones. A single SIMD instruction like 'add these two 512-bit registers' performs eight (or sixteen) additions in one shot, in roughly the time one scalar addition would take. To exploit this, the loop must be vectorizable: the iterations must be independent (no iteration depending on the previous one's result), the data should be contiguous and aligned so it loads into a lane cleanly, and there must be no branches that send different lanes down different paths. A compiler can auto-vectorize simple loops, but it gives up easily — a single data dependence, an aliased pointer, or an awkward stride and it falls back to slow scalar code.

Vectorization is one of the two ways a single core gets its peak throughput (the other being instruction-level pipelining), and it is why the same arithmetic can run 4x, 8x, or 16x faster with no change in algorithm. It is also why data layout matters so much: SIMD wants its operands packed together, which favours a structure-of-arrays layout over an array-of-structures one, and it is half the reason BLAS level-3 kernels reach near-peak speed. But SIMD only multiplies the compute side of the ledger — it does nothing for a memory-bound kernel that is already waiting on data, which is why you check the roofline first to know whether vectorizing will help at all.