GPU architecture

A GPU is a processor built around a completely different bet than a CPU: instead of a few brilliant cores that finish any single task quickly, it has thousands of simple cores that do the same operation on thousands of data items at once. A CPU is a few mathematics professors; a GPU is ten thousand schoolchildren all multiplying their own pair of numbers simultaneously. Born to colour millions of pixels in parallel, this design turned out to be perfect for any 'embarrassingly parallel' number-crunching — which is why GPUs now power scientific computing and AI.

The model is called SIMT (Single Instruction, Multiple Threads): threads are bundled into groups (warps of 32) that all execute the same instruction in lockstep on different data. GPUs hide memory latency not with big caches but with massive multithreading — when one warp stalls waiting for memory, the hardware instantly switches to another that's ready, keeping the arithmetic units always busy. The trade-off is that branchy, irregular code with lots of decisions runs poorly; GPUs shine only when the same work is done over and over on a flood of data.