average memory access time (AMAT)
/ AY-mat or A-M-A-T /
Suppose most of the time the pen is on your desk (instant), but now and then you must walk to the drawer (slow). What is your TYPICAL time to get a pen? Not the best case, not the worst case, but a blend: the fast case weighted by how often it happens, plus the slow case weighted by how often that happens. Average memory access time (AMAT) is exactly this blended, honest measure of how long a memory access takes on average, accounting for both hits and misses.
The formula is simple and worth memorizing: AMAT = hit time + (miss rate x miss penalty). The hit time is paid on every access (you always at least look in the cache); on the fraction that miss, you additionally pay the miss penalty. For a multilevel cache the penalty term itself expands recursively — the L1 miss penalty is just the AMAT of the L2, whose miss penalty is the AMAT of the L3 or memory, and so on. This single equation ties together everything that matters: how fast the cache is, how often you hit, and how badly a miss hurts.
AMAT is the right yardstick because it correctly weights rare-but-expensive misses against common-but-cheap hits — which is exactly the trap that fooling yourself with hit rate alone falls into. It also makes design tradeoffs concrete: a bigger, more associative cache raises hit time but lowers miss rate; whether that is a net win depends on which term dominates, and AMAT lets you compute it instead of guessing. Honest caveat: AMAT is an average over a model and ignores overlap, so on an out-of-order core that hides some miss latency, the AMAT formula overestimates the real slowdown — it is a clean first-order tool, not the last word on performance.
Hit time 1 cycle, miss rate 5 percent, miss penalty 100 cycles: AMAT = 1 + 0.05 x 100 = 6 cycles. Cut the miss rate to 2 percent: AMAT = 1 + 0.02 x 100 = 3 cycles — the access is now twice as fast, all from fewer misses, none from a faster hit.
AMAT = hit time + miss rate x miss penalty — the honest average that weights misses by their cost.
AMAT is an average under a simple model and assumes misses fully stall the core. On out-of-order machines that overlap misses with useful work, AMAT overstates the real slowdown — treat it as a clean estimator, not an exact run-time predictor.