the memory hierarchy

Think of where you keep things by how often you reach for them. The pen in your hand is instant. The notebook on your desk takes a second. The folder in a drawer takes longer. The box in the basement takes a real trip. None of these is wrong; you keep the most-used things closest because closeness is fast but expensive and small, while far storage is slow but cheap and huge. A computer arranges its memory the very same way.

From the OS view, the levels from fastest-and-smallest to slowest-and-largest are roughly: the CPU's registers (a few dozen words, accessed in a single clock tick), the caches (small, on or near the chip), main memory or RAM (gigabytes, electrically fast but still much slower than cache), and the disk or SSD as backing store (huge and cheap but vastly slower). The crucial fact for memory management is this: the CPU can directly read and run instructions only from registers and main memory. It cannot execute code or load operands straight off the disk. So anything the program needs to run on must first be brought into main memory.

Why it matters: this single rule shapes everything in this field. Because the CPU works only from RAM, the OS must place a program's code and data into RAM before it runs, must decide what to keep there when RAM is full, and must move things to and from disk (swapping). The hierarchy also explains why a memory reference that misses the cache and goes to RAM is already slow, and why a reference that has to go to disk is catastrophically slower — a gap of roughly a hundred thousand times or more.