3D stacking

A normal chip is flat: all its transistors live in a single thin layer of silicon, spread out side by side like houses on a single-story street. If you want more on the chip, you spread out wider, and the wires connecting distant parts get longer. 3D stacking is the move from a single-story town to a tower: you build multiple layers of silicon and stack them directly on top of one another, then drill vertical connections straight down through them so the layers can talk almost instantly. Distant components that used to be a long horizontal wire apart can now be neighbors, one floor up.

The key enabling piece is a tiny vertical wire that goes straight through the body of a silicon die, called a through-silicon via (a 'via' is just a wire that connects different layers). By etching thousands of these through-silicon vias, two stacked dies can be wired together with connections that are extremely short — a few tens of micrometers tall instead of millimeters across — and therefore very fast and very low power, with enormous total bandwidth because you can have so many of them in parallel. The flagship use is stacking memory directly on or beside a processor: high-bandwidth memory is built by stacking several DRAM dies and connecting them with through-silicon vias to feed data to a hungry GPU or accelerator far faster than flat memory chips on a distant board.

Why does this matter now? Because the dominant cost in modern computing is moving data, not computing on it, and vertical links are dramatically shorter than horizontal ones across a board. Stacking shortens the journey. The honest difficulty is heat: when you stack active, power-hungry silicon layers on top of each other, the heat from the bottom layers has to escape through the layers above it, and there is no easy path out. Managing temperature in a 3D stack is one of the hardest open problems, which is why today we more readily stack cool, dense memory than stack hot logic on hot logic.