GPU computing

A CPU is like a few brilliant chefs, each able to follow a complicated recipe quickly. A GPU is like a stadium full of line cooks, each one slow and simple, but thousands of them, all doing the identical chopping motion at once. For a fancy varied meal you want the chefs; but for peeling ten thousand potatoes the same way, the army of line cooks finishes in a flash. GPU computing is the art of recasting numerical work into that second shape — thousands of identical operations on different data — so a graphics processor's enormous parallelism can be turned loose on it.

A GPU has thousands of small arithmetic units organised for throughput, not low latency: it tolerates the long wait for memory by keeping tens of thousands of threads in flight and instantly switching to a ready one whenever others stall. Its execution model is SIMT (Single Instruction, Multiple Threads): threads are run in lock-step groups (a 'warp' of 32 in CUDA, the dominant programming framework), all executing the same instruction on different data. This is wonderful when every thread does the same thing, and wasteful when they diverge — if some threads in a warp take an if-branch and others do not, the hardware runs both paths in turn, so heavy branching kills GPU performance. GPUs also have their own memory hierarchy, including a small fast on-chip 'shared memory' that programmers manage by hand, the GPU's answer to cache blocking. A key cost is moving data across the relatively slow link between CPU and GPU memory, which can dominate if you shuttle data back and forth.

GPUs shine on exactly the kernels at the heart of scientific computing: dense linear algebra (matrix multiply maps perfectly to the regular, high-intensity, all-threads-identical pattern), stencil sweeps on grids (each grid point updated by an independent thread), and FFTs — all regular, data-parallel, high-arithmetic-intensity work. They struggle with irregular, branchy, pointer-chasing, or low-parallelism computations. The honest summary: a GPU can deliver an order of magnitude more flops and bandwidth than a CPU for the right kind of problem, but only if the work is massively and uniformly parallel, the data stays resident on the device, and the algorithm avoids branch divergence. It is throughput, not a free lunch.