high-bandwidth memory (HBM)

Picture the problem HBM was invented to solve. Your logic die — a GPU or AI accelerator — can crunch numbers far faster than it can fetch the data to crunch. That gap is the memory wall: the chip starves not because it lacks compute, but because the pipe feeding it data is too narrow. Ordinary DRAM sits on the far side of the circuit board, reached through a handful of skinny wires; widening that pipe by adding more pins runs out of room fast. HBM answers a single question — how do you make the data pipe enormously wide without making the wires long? — and its answer is to go vertical and move close.

An HBM stack is several DRAM dies (commonly 8, 12, or 16 high) piled on top of each other and wired together vertically by through-silicon vias, which are copper-filled holes punched straight down through the silicon. That tall stack is then set down right beside the logic die on a silicon interposer — a tiny circuit board made of silicon that lets you etch thousands of fine wires between the two. Instead of a narrow bus a few wires wide, HBM talks over a bus roughly a thousand bits wide, because the data only has to travel a few millimeters across the interposer rather than across a whole board. Wide-and-short beats fast-and-far: each individual wire is fairly relaxed in speed, but a thousand of them in parallel deliver staggering total bandwidth.

The trade is real estate and money for bandwidth. Stacking and TSVs are expensive to manufacture, and the silicon interposer adds cost and packaging complexity, so HBM lives where bandwidth is worth almost any price: AI training and inference, GPUs, and high-performance computing, where the model weights and activations must stream in continuously. It is the same packaging idea — known-good dies brought close together on an interposer — that powers chiplets, applied specifically to memory, turning the memory wall from a brick wall into a much wider doorway.