a cache
/ cash /
Picture a tiny desk you keep right next to you, holding only the handful of books you are actively using. The full library is across town and slow to reach, but as long as the book you want is already on the little desk, you get it instantly. When you need a book that is not there, you walk to the library once, bring it back, and place it on the desk — knocking off whichever book you have not touched in the longest. A cache is exactly this: a small, fast copy of the most recently used slices of a larger, slower memory.
A hardware cache sits between the CPU and main memory and holds copies of recently accessed memory blocks (cache lines). On every memory access the hardware first checks the cache. If the data is there it is a hit and the access is fast; if not it is a miss, and the data must be fetched from the slower level (paying the miss penalty), then stored in the cache so the next access to it is fast. Because programs have locality, the great majority of accesses are hits, so the average access time drops close to the cache's fast hit time even though the cache holds only a tiny fraction of memory.
The single most important honesty about caches: they are automatic and completely invisible to correctness. Your program produces exactly the same result whether the cache helps a lot or not at all — the cache only changes speed, never answers. But that speed difference is enormous: cache-friendly code with good locality can run many times faster than cache-hostile code computing the identical result. So a cache is best thought of as a performance feature you must EARN through locality, not a free gift. Modern CPUs have several cache levels (L1, L2, L3) for this reason.
Read address A: miss, so the line containing A is fetched from DRAM (~80 ns) and stored in the cache. Read A again, and the neighbours that came with it: hits (~1 ns each). The first access paid the penalty; the rest ride free on locality.
A small fast copy of recently used memory; hits are quick, misses pay to refill, locality makes hits common.
A cache never changes what your program computes — only how fast. Two runs with very different cache behaviour give identical results. This is why cache bugs show up as mysterious slowness, not wrong answers (within a single core; cross-core sharing is the separate topic of coherence).