branch divergence

Picture a rowing crew of 32 who must all pull the same stroke at the same instant because they share one drumbeat. Now give them an instruction that some interpret as 'row left' and others as 'row right'. They cannot do both at once with a single shared beat, so the coxswain must say: 'left-rowers go, right-rowers freeze' — then 'right-rowers go, left-rowers freeze.' The work that should have taken one beat now takes two, with half the crew idle each time. That, on a GPU, is branch divergence: when threads in one warp take different sides of a branch, the hardware must execute the paths one after another.

Concretely, all threads in a warp share one program counter and instruction-fetch unit, so they must execute the same instruction each step. When code says if (x > 0) do A else do B, and within a warp some threads have x > 0 and some do not, the hardware cannot run A and B simultaneously. It runs path A with a mask that disables the else-threads (they sit idle), then runs path B with a mask that disables the if-threads. The two paths are serialized; the warp's effective throughput for that region drops by the fraction of threads that were masked off. Deeply nested or data-dependent branches can stack this penalty.

This is the single most important performance pitfall a GPU beginner must internalize, and it is why a GPU is honestly bad at branchy code. Note what divergence is not: a branch where all 32 threads in a warp agree (all take the if, or all take the else) costs nothing extra — the warp simply runs one path, no idling. The penalty comes only from disagreement within a warp; branches at coarser granularity, where whole warps go one way, are fine. Good GPU code therefore tries to arrange data so that nearby threads (those in the same warp) tend to take the same path — for example sorting work by branch outcome — turning divergence back into agreement.