a throughput processor
Two restaurants. One is a fine-dining kitchen with a single virtuoso chef who plates each dish as fast as humanly possible — you, the lone diner, get your meal in record time. The other is a cafeteria line with twenty cooks each doing one simple step; any single tray takes a while to traverse the line, but the kitchen serves a thousand trays an hour. The virtuoso is a latency-optimized processor (a CPU); the cafeteria is a throughput processor. A throughput processor is built to maximize total work finished per second across many tasks, even at the expense of how fast any one task finishes.
The two designs make opposite hardware bets. A latency-optimized CPU spends its transistors making a single thread fast: big caches to avoid waiting on memory, deep out-of-order execution, branch prediction, high clock speed — all to shorten the path of one instruction stream. A throughput processor (the canonical example is a GPU) spends its transistors on many simple cores and many in-flight threads instead. It does not fight memory latency with big caches; it hides it by always having another ready thread to run while one waits — so a memory stall that would idle a CPU just means the throughput machine switches to other work. Width and occupancy, not single-thread speed, are its currency.
The honest framing matters because the two are not ranked, they are specialized. A throughput processor is wonderful when you have thousands of independent tasks (pixels, array elements, matrix entries) and terrible when you have one long dependent chain — there is no second thread to hide behind, and each step is individually slow. This is the same truth as 'a GPU is not just a faster CPU', stated in architectural terms. Real systems are heterogeneous: a latency-optimized CPU runs the serial, branchy control logic and hands the regular, parallel bulk to a throughput processor. Choosing the wrong one for a workload — serial code on a GPU, or massively parallel number-crunching stuck on a CPU — wastes most of the hardware.
Same chip budget, two philosophies: a CPU might spend it on 8 powerful cores with huge caches and deep out-of-order machinery (great for one fast thread); a GPU spends it on thousands of slim cores with little cache but room for tens of thousands of threads (great for total work). On a serial task the CPU wins; on a million-element parallel task the GPU wins by a wide margin.
A throughput processor hides latency with many threads instead of fighting it with big caches; its goal is total work per second, not single-task speed.
Throughput-optimized is not 'better than' latency-optimized — it is specialized. Put a serial, dependent task on a throughput processor and it crawls, because the trick of hiding latency behind other threads has no other threads to use.