the GPU and TPU
/ GEE-pee-yoo and TEE-pee-yoo /
A GPU (graphics processing unit) and a TPU (tensor processing unit) are the specialized chips that do the heavy arithmetic of machine learning. The ordinary processor in your laptop, the CPU, is like a brilliant lone chef who can cook any dish but only one pan at a time. A GPU is more like a vast kitchen of thousands of line cooks, each doing one simple step in parallel. Neural networks are mostly the same simple operation — multiplying and adding big grids of numbers — repeated millions of times, which is exactly the kind of work a GPU was built to spread across all those cooks at once.
GPUs were originally designed to draw video-game graphics, which is also a flood of parallel number-crunching, and researchers discovered around 2009–2012 that the same hardware trained neural networks dramatically faster. A TPU is Google's chip built from scratch for this one job: it strips away the graphics machinery and packs in circuitry that does the dense matrix multiplication at the heart of neural networks even more efficiently. Other companies make their own accelerators too; GPU and TPU are just the two most familiar names.
Why it matters: the modern AI boom is as much a hardware story as a math story. Models that would take months to train on CPUs train in days on clusters of accelerators. But this comes with hard limits — these chips are expensive, hungry for electricity, often in short supply, and constrained by how much fast memory they hold. A model that does not fit in a chip's memory simply cannot run on it without extra tricks. Picking and feeding this hardware well is a craft in itself, and it is often the real bottleneck, not the cleverness of the algorithm.
Multiplying two 1000×1000 grids of numbers takes a billion multiply-add steps. A CPU with a handful of cores chews through them in small batches; a GPU with thousands of cores fires off thousands at once — the same job, finished in a fraction of the time.
Same arithmetic, vastly more lanes — why parallel chips dominate neural-network work.
A faster chip does not automatically mean a faster model. If the chip sits idle waiting for data to arrive from memory or from disk, you are 'memory-bound,' and a more powerful processor buys you nothing. Real-world performance is a tug-of-war between raw compute and how fast you can feed it.