Why one big chip stopped being the answer
The first guide of this rung explained the bind we are in: Dennard scaling ended around 2005 and Moore's law is slowing, so we can no longer just print smaller, faster transistors and wait. One response, covered earlier, was the SoC — pack a CPU, a GPU, and accelerators onto one die. But that very strategy has hit a wall of its own, and the culprit is brutally physical: yield. When you make a wafer, defects fall on it more or less at random, and the bigger a single die is, the more likely it is to catch a fatal defect. Double the area and you do not double the cost — you more than double it, because so many large dies come out broken.
A small worked picture makes this vivid. Suppose a wafer averages one defect per 100 square millimetres of area, scattered randomly. A 100-square-millimetre die expects roughly one defect, so a large fraction come out dead. Now cut that same logic into four 25-square-millimetre chiplets: each tiny piece is far more likely to be defect-free, you keep the good ones, throw away only the few bad small ones, and bin chiplets by speed. The total silicon is the same, but far more of it is sellable. This is the core economic argument: small dies have dramatically better yield, and yield is what sets the price.
There is a second motive, and it is just as practical. Not every part of a processor wants to be made on the newest, most expensive manufacturing process. Fast logic cores benefit enormously from the latest node, but I/O circuits, analog blocks, and large cache arrays barely improve and cost a fortune there. Splitting them into separate chiplets lets you build each one on the process that suits it — bleeding-edge silicon for the cores, a cheaper, older, more reliable process for the I/O — and assemble the mix afterward. One monolithic die forces a single compromise process on everything at once; chiplets let each block pick its own.
What a chiplet actually is
A chiplet is simply a small, separately-manufactured die that does one part of the job — a cluster of CPU cores, an I/O hub, a memory controller — and is designed to be joined to other chiplets inside a single package. Think of it as moving from building a house as one enormous poured slab to building it from prefabricated rooms trucked in and bolted together. The rooms are the chiplets; the foundation and corridors that connect them are the package. The processor a programmer sees is the assembled building, not any one room.
The catch is that the rooms must agree on how their corridors meet. Two chiplets only work together if they speak the same physical and logical interconnect — same wire layout, same signalling, same protocol — at the boundary where they join. That is why the industry has been hammering out shared die-to-die standards (UCIe is the prominent one): an open agreement for the seam, so a chiplet from one vendor can plug against a chiplet from another. It is the same idea as an open ISA from the previous guide, pushed down a level — instead of agreeing on the software contract, here we agree on the contract between pieces of silicon. Standardize the seam and a whole marketplace of mix-and-match chiplets becomes possible.
Stitching them side by side: 2.5D and the interposer
The first way to join chiplets is to lay them flat, side by side, but on a far richer surface than an ordinary circuit board. That surface is a silicon interposer: a thin slab of silicon, with no logic of its own, whose only job is to carry an enormous density of tiny wires between the chiplets sitting on top of it. Because it is silicon, it can pack thousands of connections in the space a board would give you a handful — the corridors between the prefab rooms become wide, short, and numerous instead of long and cramped. Stacking chiplets in one plane on top of an interposer is what the industry calls 2.5D integration: not a single flat die, not yet a true vertical stack, but something in between.
package substrate (to the board)
+-----------------------------------------------+
| +---------+ +---------+ +-------------+ |
| | CPU | | CPU | | HBM stack | |
| | chiplet | | chiplet | | (DRAM dies) | |
| +----o----+ +----o----+ +------o------+ |
| | thousands of fine wires | |
| ===== silicon interposer (passive) ======= |
+-----------------------------------------------+
2.5D: chiplets side by side on one interposerThe marquee use of 2.5D is the one you already met in the accelerator guides: placing high-bandwidth memory right beside a GPU or accelerator on the same interposer. Recall the recurring enemy — the memory wall, the gap between how fast a processor can compute and how fast memory can feed it. HBM attacks it by putting the DRAM millimetres away over thousands of interposer wires, giving a colossally wide path to memory. That wide path only exists because the interposer can route a number of connections an ordinary package never could; HBM and 2.5D packaging grew up together precisely to widen the doorway into memory.
Stacking them upward: true 3D and TSVs
If 2.5D is rooms side by side, 3D stacking is rooms stacked into a tower. Here dies sit directly on top of one another, bonded face to face, and the trick that makes it possible is the through-silicon via (TSV): a tiny vertical wire drilled straight down through the body of a die so the layer above can talk to the layer below. Going up rather than out shortens the longest wires dramatically — a signal that crossed the whole chip now drops a few microns to the floor below — which cuts both latency and the energy spent moving data, the bill that earlier guides showed dominates a modern chip.
You have, in fact, already been using 3D stacking. An HBM module is itself a vertical stack of several DRAM dies bonded together with TSVs — that is the stacked memory that makes its bandwidth possible. And the frontier now reaching products is stacking compute on compute: bonding an extra slab of cache directly on top of CPU cores so a much larger, faster cache sits a few microns away rather than off to one side. The same principle reappears whenever bandwidth or latency between two blocks is the limit: stop sending data across the chip and stack the blocks so the data barely has to move at all.
The hard parts, and why this matters
Be honest about what 3D stacking costs, because the physics fights back. The worst enemy is heat. When you stack dies, you trap the heat of the lower layers under the upper ones, and a hot transistor is a slow, leaky, failure-prone transistor. This is why you rarely see the hottest, highest-power logic stacked directly on more hot logic; it is far easier to stack cool, dense memory on top of compute, or compute on top of a cooler I/O layer. Stacking also makes testing harder — a stack is only as good as its worst die, and a single bad layer can ruin an otherwise perfect tower, a fresh twist on the chiplet yield story.
There is also a subtler trap: a chiplet design only pays off if the communication between chiplets does not eat the savings. Crossing from one die to another, even over a dense interposer, is slower and more power-hungry than staying on one die, so a workload that constantly shuttles data between chiplets can run worse than the monolithic version it replaced. The architect's job is to draw the cut lines where traffic across them is naturally low — keep tightly-coupled blocks together on one chiplet, split only where the boundary is quiet. It is the same locality reasoning as false sharing in the multicore guides, lifted up to the level of whole dies: put things that talk a lot close together.
Step back and the significance is large. In the post-Moore era, when shrinking transistors no longer hands you free gains, advanced packaging has become a genuine new axis of improvement — a way to keep building bigger, faster systems by assembly rather than by lithography alone. It is no accident that this sits in the same rung as specialization and open ISAs: all three are the field's answer to the same exhausted scaling curve, finding gains in how we compose and connect silicon rather than in the silicon getting magically faster. The processor of the next decade is less a single carved stone and more an assembled structure — and knowing how it is bolted together is now part of the craft.