What we are even measuring
By now the iron law from guide 1 is in your bones: a program's run time is instruction count x CPI x cycle time, and each layer — compiler, ISA, microarchitecture, silicon — moves one of those factors. So the cleanest measure of performance is brutally simple: how long did the actual program take, wall-clock, start to finish? Everything fancier is a shortcut, and every shortcut is a place to fool yourself.
The first fork is latency versus throughput, which guide 1 also drew. Latency is how long one task takes from request to result; throughput is how many tasks finish per second. They are different questions and a change can help one while hurting the other. Deepening a pipeline raises throughput but adds a touch of latency per instruction; batching web requests raises throughput but makes any single user wait longer. Before you quote a single number, decide which one you are actually paying for — a video encoder lives on throughput, a game's input loop lives on latency.
Why MIPS and FLOPS lie to you
It is tempting to rate a CPU by raw speed: MIPS (millions of instructions per second) or FLOPS (floating-point operations per second). Both sound objective and both can mislead badly. MIPS = instruction count / (run time x 10^6), which means MIPS = clock rate / (CPI x 10^6). Stare at that and the trouble jumps out: it rewards executing more instructions per second, but a program is faster when it executes fewer instructions overall. A CISC machine and a RISC machine doing the same job can post wildly different MIPS numbers while finishing in the same time — because they need different instruction counts. MIPS measures effort, not result.
There is a darker version of the trap. Suppose a clever compiler optimization deletes redundant work, so the program runs 20% faster — but the instructions it deleted were cheap, high-MIPS ones. The run time drops, yet the MIPS rating also drops, because now the surviving instructions are the slow ones. By the MIPS yardstick the faster machine looks worse. People used to joke that MIPS stood for 'Meaningless Indicator of Processor Speed', and they were not entirely joking.
FLOPS has the same disease in a lab coat. It is genuinely useful in a narrow world — dense linear algebra, where the floating-point work really is the job — but quoting 'peak FLOPS' for a whole machine usually means the theoretical maximum nobody ever reaches: every floating-point unit busy every cycle, no stalls, no waiting on memory. Real code sits far below peak, and the gap is itself a fact about the architecture (it is what the roofline model you will meet later exists to expose). Peak FLOPS tells you what the machine cannot exceed, not what your program will get.
Real benchmarks, and the mean that does not cheat
If single numbers lie, the honest answer is to run a suite of real programs and report all of them. That is what a benchmark is — and the industry-standard collection is SPEC, a curated set of real applications (compilers, video, physics, AI) that vendors run under strict rules so results are comparable. A suite resists gaming: it is hard to special-case a dozen diverse programs at once, and SPEC's committee retires benchmarks once compilers learn to cheat on them. But a suite gives you many numbers, and sooner or later someone wants one summary. That is where the real trap springs.
Suppose you measure each program as a speedup ratio — how many times faster machine A is than a reference. To summarize, the obvious move is the arithmetic mean (add them up, divide by how many). It is wrong for ratios, and dangerously so: the arithmetic mean of ratios depends on which machine you picked as the baseline. Flip the reference and you can flip which machine 'wins'. A summary that changes its verdict when you swap the denominator is not a summary; it is a coin you can weight.
Two programs, two machines. Speedup of A relative to baseline B:
prog 1 prog 2 arithmetic mean geometric mean
A vs B : 4.0 0.5 2.25 sqrt(4.0*0.5)=1.41
B vs A : 0.25 2.0 1.125 sqrt(0.25*2.0)=0.71
Arithmetic mean: A looks 2.25x, B looks 1.125x -> who is faster
flips with the baseline. Geometric means are reciprocals
(1.41 vs 0.71), so the verdict is stable: A is sqrt(2)~1.41x B.The fix is the geometric mean: multiply the n ratios together and take the n-th root. It has the one property a ratio-summary must have — it is consistent no matter which machine is the baseline, because the geometric mean of A-over-B is exactly the reciprocal of the geometric mean of B-over-A. That is precisely why SPEC reports a geometric mean of normalized speedups. It is not a matter of taste; it is the only mean that refuses to let the choice of reference rig the answer.
Amdahl's law: the ceiling on every speedup
Now suppose you make one part of a program faster — a new vector unit, more cores, a hand-tuned hot loop. How much does the whole program speed up? The answer is humbling and is called Amdahl's law. If a fraction f of the run time is sped up by a factor s, while the remaining (1 - f) is untouched, the overall speedup is 1 / ((1 - f) + f/s). The whole program can never go faster than the part you did not improve allows.
Push s to infinity — make the improved part instant — and the speedup tops out at 1 / (1 - f). So if 90% of a program parallelizes perfectly but 10% stays serial, even with infinite cores you get at most 1 / 0.1 = 10x, never more. Throw a thousand cores at it and you are still capped near 10x while paying for a thousand cores. This is the cold arithmetic behind a warning from earlier rungs: more cores rarely means proportional speedup, because Amdahl's law makes the stubborn serial fraction the boss.
Power, energy, and choosing the metric that matters
Speed is only half the story; the next two guides make power the centerpiece, so meet the distinction now. Power is rate — joules per second, watts — and it sets how hot the chip runs and how big the cooler must be. Energy is the total — power integrated over time, joules — and it sets your battery life and your electricity bill. They pull apart: running at half the clock might halve power yet, if the job takes twice as long, use the same energy. A phone cares about energy (battery); a datacenter cares about both (the power bill and the cooling wall).
Because neither speed nor power alone captures 'good', architects often quote the energy-delay product (energy x time): it punishes a design for being either wasteful or slow, so you cannot game it by burning more watts to shave a millisecond, nor by crawling along to sip power. There is no universal 'best' metric — and that is the honest core of this guide. The right number depends on what you are paying for: latency for a trader, throughput for a render farm, energy for a watch, energy-delay for a laptop that must be both quick and cool.
One last guardrail against fooling yourself. Always report what the workload was, run each test enough times to see the variance (modern machines are noisy — caches warm up, the clock boosts then throttles, the OS interrupts), and never trust a number you cannot reproduce. The reason power even forces its way into a chapter about speed is the end of Dennard scaling around 2005 — clocks stopped getting free speedups — which is exactly the story the last two guides in this rung tell.