the storage hierarchy

Imagine organising a kitchen by how often you reach for things. Salt and a knife sit on the counter, within arm's reach. Pots live in a low cupboard, a step away. The big stockpot is in the basement. Nobody puts everything on the counter — there is not room, and counter space is precious — so you keep the most-used items closest and accept a longer walk for the rare ones. A computer organises its memory and storage in exactly this layered way, trading speed against size and cost.

Concretely, the levels run from fastest-smallest-priciest at the top to slowest-largest-cheapest at the bottom: CPU registers (a few words, a fraction of a nanosecond), CPU caches (kilobytes to megabytes, a few nanoseconds), main memory or RAM (gigabytes, around 100 nanoseconds), and mass storage — SSD then HDD — at the bottom (terabytes, microseconds for an SSD, milliseconds for a disk). Each step down is roughly an order of magnitude or more slower but far bigger and cheaper per byte. The single dividing line that matters most: everything above the storage level is volatile (lost on power-off) and the CPU can work from it directly; mass storage at the bottom is non-volatile but the CPU cannot execute from it.

Why it matters: the entire job of memory management, caching, and virtual memory is to hide this hierarchy so the program feels like it has one fast, infinite memory. Locality of reference — the fact that programs tend to reuse the same data and nearby data — is what makes the trick work: keep the hot stuff high in the hierarchy and most accesses land on the fast levels. The same hierarchy now has new rungs, such as persistent memory, that blur the old clean line between volatile RAM and persistent storage.