Several desks, one library
In the previous guide we saw why the chip industry turned to many cores: the power wall stopped single cores from getting much faster, so the only way forward was to put several cores on one die and run threads in parallel. The most common arrangement is the shared-memory multiprocessor: every core sees the same main memory, the same addresses, as if they were all writing in one shared notebook. That shared view is exactly what makes thread-level parallelism pleasant to program — pass a pointer, and any core can follow it.
But recall the lesson of the caches rung: a core that walked to main memory on every access would crawl. Each core therefore keeps its own private cache — a small fast desk holding its most-used cache lines so it rarely has to walk to the library. This is wonderful for one core in isolation. With several cores, it quietly plants a contradiction: the same address can now sit, simultaneously, on several private desks. As long as everyone only reads, that is harmless — identical photocopies of one page. The trouble starts the instant someone picks up a pen.
When the photocopies disagree
Picture a shared variable `x`, currently 0, sitting in main memory. Core A reads it, so a copy of `x = 0` now lives in A's cache. Core B reads it too, so `x = 0` lives in B's cache as well — two photocopies, both correct. Now Core A executes `x = 1`. If A uses a write-back policy, it scribbles the new value onto its own desk and does not immediately walk to the library. A's cache now says `x = 1`. But B's cache still says `x = 0`, and so does main memory. Three copies of one address, and they no longer agree. The next time B reads `x`, it will confidently hand back a stale 0 — a value that is, by now, simply wrong.
This is the cache coherence problem in one breath: with private caches over shared memory, a write by one core can leave other cores holding stale copies, so different cores observe different values for the same address. Notice it is not a bug in anyone's program — both cores ran perfectly ordinary loads and stores. It is the hardware that has silently broken the shared-notebook illusion that made the programming model sane in the first place. If the machine is to keep its promise, the hardware itself must repair this.
The fix in one idea: tell the others
The repair, at its heart, is communication. Before a core is allowed to modify a line, it must make sure no stale copy of that line survives anywhere else. The dominant strategy is write-invalidate: when Core A wants to write `x`, it first announces 'I am about to write this line — everyone else, throw your copy away.' Every other core that holds the line invalidates it. Now A is the only holder, free to scribble. When B next reads `x`, it finds nothing on its desk, suffers a cache miss, and is handed the fresh value (from A's cache or from memory). The stale 0 can never be returned, because it no longer exists.
How does the announcement reach everyone? Two families of protocol answer differently. In a snooping protocol, the cores share a broadcast medium — classically a bus — and every cache controller listens in on every transaction, like neighbours overhearing a shouted message. When A broadcasts 'invalidate `x`', B's cache hears it directly and acts. Snooping is beautifully simple but assumes everyone can hear every shout, which stops scaling once you have many cores. The alternative, a directory protocol, keeps a directory that records, for each line, exactly which cores hold a copy; A's write sends targeted invalidations only to those cores, no broadcast needed. Directories cost more bookkeeping but scale to the large machines where shouting would drown everyone.
To make all this trackable, each line in a cache carries a little state, and the controller runs a tiny state machine on it. The most famous scheme is MESI, whose four states — Modified, Exclusive, Shared, Invalid — let a cache know whether its copy is the only one, whether it is dirty, and whether it may be silently read. We will trace MESI's full state dance in the next guide; for now it is enough to know that 'remember the status of each line, and talk before you write' is the whole game.
False sharing: the trap that punishes the innocent
Here is the subtlety that bites real programmers. Coherence is tracked not byte by byte but at the granularity of a whole cache line — typically 64 bytes. The hardware can only invalidate or share entire lines, never half of one. So two variables that have nothing logically to do with each other but happen to land in the same line are, to the coherence machinery, indistinguishable from the same variable. Touch one, and you disturb the other's line. This is false sharing, and it is one of the cruelest performance traps in parallel code, because the program is correct — it just mysteriously crawls.
Picture the classic case: an array `counts[2]`, where Core 0 hammers `counts[0]` in a tight loop and Core 1 hammers `counts[1]`. Logically these are independent — no shared data, no lock needed. But both ints sit in one 64-byte line. Every time Core 0 writes `counts[0]`, write-invalidate forces Core 1's copy of the whole line to be thrown away; Core 1's next write then yanks the line back and invalidates Core 0's. The line ping-pongs between the two caches on every single update, each one a coherence miss costing dozens of cycles. Two cores that share nothing real end up serializing, slower than one core would have been.
Address line (64 bytes), bytes 0..63:
[ counts[0] | counts[1] | ... unused ... ] <- ONE cache line
^Core0 ^Core1
Core0: write counts[0] -> invalidate the line in Core1
Core1: write counts[1] -> invalidate the line in Core0
... line ping-pongs every write (a coherence miss each time)
Fix: pad so each counter sits in its own line:
struct { long v; char pad[56]; } counts[2]; // 64-byte alignedCoherence is necessary, but it is not free — and not everything
Be honest about what coherence buys and what it costs. It buys correctness: with a working protocol, ordinary loads and stores on shared data return sane, agreed-upon values, and the shared-notebook illusion holds. But it costs traffic and latency. Every write to a shared line must broadcast or send invalidations; every subsequent read by another core pays a coherence miss to refetch. A line bouncing between caches — whether from true sharing or false — can be far slower than a normal miss to memory. This is the same recurring architecture lesson: a cache's benefit is earned through locality, and coherence-hostile sharing patterns can leave parallel code many times slower while computing the identical answer.
Two limits are worth fixing in your mind now. First, coherence guarantees that writes to one address become visible and ordered — it does not by itself tell you how writes to different addresses interleave as seen by other cores. That looser, subtler guarantee is the memory consistency model, and relaxed models will surprise you; we take it up later in this rung. Second, on a big machine memory itself is not uniform: under NUMA, each core reaches its nearby memory bank quickly and a distant bank slowly, so where a shared line physically lives changes the cost even after coherence has done its job. Coherence keeps the values correct; it does not make every access equally cheap.
Step back and the shape of the problem is clear. Private caches gave each core speed; shared memory gave us an easy programming model; and the two together demanded a hardware referee — the coherence protocol — running quietly under every memory access on a multicore chip. In the next guide we follow that referee in detail, tracing the MESI states as a line moves between Modified, Exclusive, Shared, and Invalid, and watching exactly what message each transition sends.