Cache memory

A cache is a small, blazing-fast memory that sits right next to the processor and holds copies of the data it has used most recently — like keeping the few tools you reach for constantly on the desk in front of you instead of walking to the storeroom (main memory) every time. The processor checks the cache first; a hit returns data in a couple of cycles, while a miss means a slow trudge to DRAM, hundreds of cycles away.

Caches work because real programs exhibit locality: they reuse the same data soon (temporal) and access nearby addresses next (spatial), so a cache fetches whole 64-byte lines at a time. Chips stack them in a hierarchy — a tiny, instant L1 (32 KB), a larger L2 (256 KB–1 MB), and a big shared L3 (tens of MB) — each bigger and slower than the last. Because main memory is roughly 100× slower than the core, cache hit rates above 95% are what keep a processor from starving; closing that speed gap is the whole job of the 'memory wall'.