The free lunch that lasted fifty years
For most of computing history, a programmer who wanted faster code could simply wait. Every couple of years a new chip arrived that ran the same program noticeably quicker — no rewrite, no effort, just patience. Behind that magic stood two famous observations. Moore's law noticed that the number of transistors you could pack onto a chip kept doubling roughly every two years, because each generation shrank the transistors. More transistors meant room for bigger caches, deeper pipelines, more clever tricks. That is the half most people have heard of.
The less-famous but equally crucial half was Dennard scaling. It said that as transistors shrank, the power density stayed roughly constant — smaller transistors used proportionally less voltage and current, so you could pack more of them in and clock them faster without the chip melting. This is why clock rates climbed from megahertz to gigahertz through the 1990s and early 2000s. Moore's law gave you more transistors; Dennard scaling let you run them faster and cooler at the same time. Together they handed the industry decades of free, automatic speedup.
It is worth being precise about what these 'laws' even are. Neither is a law of physics like gravity — both are empirical trends, observations about what engineering kept managing to deliver. That distinction matters enormously, because a trend can end, and one of these did. Moore's 'law' was really a self-fulfilling industry roadmap; Dennard scaling was a consequence of how transistors behaved, right up until they got so tiny that the assumptions broke.
Why the lunch ended: Dennard broke first
Around 2005, Dennard scaling effectively ended — and this, not Moore's law, is the real turning point of modern architecture. As transistors shrank past a certain size, the voltage could no longer drop in step, and leakage current crept in: the transistor now wasted power even when it was supposed to be off, the way a tap that never quite closes keeps dripping. Push the clock higher and the power and heat shot up roughly with the cube of the frequency. Chips hit a hard thermal ceiling: you simply cannot pull the heat out fast enough. This is the power wall, and it is why clock rates stalled near 3 to 4 GHz around 2005 and have barely moved since.
Here is the cruel twist. Moore's law kept giving more transistors for years after Dennard scaling died — but Dennard's death meant you could no longer power and cool all of them running flat-out at once. The slang for this is dark silicon: a chip full of transistors of which a growing fraction must sit idle (dark) at any moment, or the whole thing overheats. The old strategy of spending new transistors on one ever-bigger, ever-faster core suddenly made no sense — a bigger core just burns more of your fixed power budget. Architects needed an entirely different plan for what to do with all those transistors.
The first answer you already met several rungs ago: multicore. If you cannot make one core much faster within the power budget, put several moderate cores on the chip and run work in parallel. This is the reason your laptop has 8 cores instead of one 8-GHz monster — multicore is not a luxury, it is the direct child of the power wall. But parallelism is not free speedup, and the limit on it has a name we will keep meeting: Amdahl's law.
The iron law still rules, and Amdahl sets the ceiling
Before chasing new tricks, anchor on the one equation that never changes — the iron law of performance. The time a program takes is always: run time = instruction count x CPI x cycle time. There is no fourth factor and no escaping it. A faster clock shrinks only the last term — which is exactly why a higher clock rate alone does not mean a faster machine. This is the megahertz myth: a 4 GHz chip that needs more instructions, or stalls more cycles per instruction, can lose to a 3 GHz chip that does less work. Every gain we discuss in this rung is really an attack on one of these three terms.
Now the ceiling on parallelism. Amdahl's law says that if a fraction of your program is inherently serial — it cannot be split across cores — then that part caps your total speedup no matter how many cores you throw at it. Suppose 95% of the work parallelizes perfectly but 5% is stuck serial. With infinite cores the parallel part shrinks to nothing, but the serial 5% remains, so the most you can ever go is 1 / 0.05 = 20 times faster. Not 100 cores' worth, not 1000 — twenty, forever. Double the cores from 16 to 32 here and you barely notice. This is why more cores rarely means proportionally more speed.
Where gains come from now: specialize, and co-design
With the clock frozen and dark silicon forcing most transistors to sit idle, the modern answer is to spend them on hardware that is idle most of the time but devastatingly fast when it runs — a domain-specific architecture. Instead of one general-purpose core trying to do everything passably, you build dedicated circuits for one important job: a video decoder, an encryption engine, or a matrix-multiply unit for neural networks. A general core is a Swiss-army knife; a specialized unit is a single perfect tool. Because it does only one thing, it can strip away the overhead a general core carries and finish the same work using a fraction of the energy.
The trade-off is honest and sharp: a specialized unit is brilliant at its job and useless at everything else. So a modern chip is a patchwork of many such units around a few general cores — a heterogeneous system. The biggest single driver of this whole shift has been machine learning, whose appetite for one operation — multiplying big matrices — is so enormous that purpose-built accelerators (Google's TPU, the tensor cores in a GPU) now do most of the heavy lifting. ML is to the post-Moore era what spreadsheets were to the PC: the killer workload that justifies new silicon.
Squeezing the most from specialized hardware demands a second idea: hardware-software co-design. In the free-lunch era, hardware and software teams could ignore each other — the chip just got faster underneath whatever code ran on it. That contract is broken. An accelerator only shines if the software is rewritten to feed it the right shape of data, and the hardware is in turn shaped around the workloads that matter. Architects now design the hardware-software interface and the algorithms together, in a tight loop. The wins no longer fall from the sky; you have to engineer both sides to meet.
The frontiers ahead, and one honest warning
The rest of this rung is a tour of where these pressures are pushing architecture. We will see the open-ISA movement and RISC-V, a free, public instruction set anyone can build on instead of paying to license one. We will see chiplets and 3D stacking — when you cannot make one big chip faster, glue several smaller ones together cleverly. We will meet processing-in-memory, which fights the von Neumann bottleneck — that single doorway every instruction and datum squeezes through — by computing right where the data lives instead of dragging it across the chip. And we will look honestly at non-traditional models: neuromorphic, analog, optical, and quantum.
Through every one of these frontiers, one thing does not change — the craft of architecture itself. Technologies come and go, but the tradeoff-driven thinking endures: the iron law tells you what time is really made of; Amdahl warns you the serial part will bite; make the common case fast tells you where to aim; locality, pipelining, and balance recur in every new substrate. A quantum or neuromorphic device still obeys an analogue of Amdahl's law and still rewards making the common case fast. Learn the timeless craft, and you can reason about a machine that has not been invented yet.