From the problem to a rulebook
Guide two left us with the cache coherence problem in its sharpest form: every core has its own private cache, so the same memory location can sit in several caches at once, each a separate photocopy of one shared page. The moment one core writes its copy, every other copy is silently stale — and a later read on another core would hand back an old value as if nothing happened. A coherence protocol is the rulebook that forbids exactly this. Its promise, informally, is simple: a read always returns the value of the most recent write to that location, no matter which core did the writing.
To enforce that promise without making every memory access crawl, two design choices matter. First, what do you do when a core writes a shared line? You could push the new data out to every other copy (write-update), or you could tell every other copy to throw itself away (write-invalidate) and let them re-fetch later if they still care. Almost all real machines pick write-invalidate, because a burst of writes to one line then costs only a single invalidation instead of a flood of updates. Second, how does a core find out a write happened? That is where snooping comes in.
Snooping: the shared bus as a gossip channel
In a small shared-memory machine, the caches all hang off one shared bus to memory — a single set of wires everyone can hear. Snooping exploits exactly that: every cache controller listens to (snoops on) every transaction on the bus, not just its own. When core 0 needs to write a line it does not just write quietly — it first broadcasts an intention onto the bus, in effect shouting "I am about to modify address X." Every other cache hears it, checks whether it holds X, and if so reacts by the rules. No central referee is needed; the bus itself is the gossip channel through which everyone stays informed.
This is beautifully simple, and it is why snooping dominates for small core counts. But notice the catch hiding in the word broadcast: every coherence message is heard by everyone, so the bus is a shared resource that all cores contend for. As you add cores, that one set of wires becomes a bottleneck — the very one doorway problem, now for coherence traffic. Snooping scales gracefully to maybe a dozen or a few dozen cores; past that, broadcasting to everyone is too expensive, and machines switch to a directory protocol that keeps a per-line list of exactly which caches hold a copy, so a write notifies only those few, point to point, instead of shouting to the whole room.
MESI: four letters that track who holds a copy
Snooping tells a cache when something happens; it still needs to know how to react. That logic lives in a tiny state machine attached to every cache line — and the classic one is MESI, named for the four states a line can be in. Think of them as four honest labels a core sticks on its photocopy, recording how much it can trust that copy and what it owes the others.
MESI states for one cache line
M Modified I am the ONLY copy, and I have changed it.
Memory is stale; I must write back before others read.
E Exclusive I am the ONLY copy, and it still matches memory (clean).
S Shared I have a clean copy; OTHER caches may also have it.
I Invalid My copy is junk; a real read/write must miss and re-fetch.
The two questions each state answers:
am I the only owner? have I modified it?
M yes yes
E yes no
S no (maybe shared) no
I --- --- (no valid data)Why four and not two? The clever ones are E (Exclusive) and the M/S split. Exclusive means "I have the only copy and it is clean" — so if this core now writes, it knows for certain nobody else holds the line, and it can silently jump to Modified without broadcasting anything at all. That single optimisation is huge for the extremely common pattern of one core reading then writing its own private data: the read brings the line in as E, the write upgrades it to M for free, and the bus never carries a coherence message. Shared versus Modified captures the other axis — whether others might have a copy (so a write must invalidate them first) and whether memory is out of date (so the line must be written back before anyone else can read it).
Walking a write through the states
Let us trace one concrete scene, the same kind guide two used: two cores sharing a counter that lives on one cache line. Watch how the labels move and how snooping carries the news. The key transitions to feel are the invalidate broadcast on a write, and the write-back-on-demand when someone tries to read a line another core has modified.
- Core 0 reads the counter; no one else has it. The line loads from memory and core 0 labels it E (Exclusive) — sole owner, clean.
- Core 0 writes (counter++). Because it was Exclusive, it knows it is alone — it flips silently to M (Modified) with no bus message at all. Memory is now stale.
- Core 1 now reads the same counter. Its cache misses and it broadcasts a read request on the bus. Core 0 snoops this, sees it holds the line in M, and intervenes: it supplies the fresh value and writes it back to memory. Both cores now hold the line as S (Shared).
- Core 1 wants to write the counter. It holds the line only as Shared, so it must first broadcast an invalidate. Core 0 snoops it and drops its copy to I (Invalid). Only then does core 1 upgrade its line to M and perform the write.
- If core 0 reads the counter again, its copy is Invalid, so it misses, broadcasts, and core 1 (now in M) supplies the freshest value — and the cycle repeats. That repeated bouncing of one line between two cores is exactly the cost coherence makes you pay for sharing.
Read that last line again, because it is the whole point. Coherence is not free — it is correct, but every time two cores take turns writing a shared line, the line ping-pongs between their caches, each turn costing a cross-core transfer of dozens of cycles. The protocol never lets you read a stale value; it simply makes genuine sharing expensive. That is exactly the right trade, and it sets up the next two ideas: false sharing (paying this cost by accident) and synchronization (paying it on purpose, correctly).
False sharing: paying the bill by accident
Now the trap. The coherence protocol works on whole cache lines — typically 64 bytes — never on individual variables. It does not, and cannot, see that two cores are touching different bytes; it only sees that they are touching the same line. So imagine an array `count[NUM_CORES]` where each core updates only its own slot. Logically there is zero sharing — no two cores ever touch the same counter. But if those slots are small and packed contiguously, several of them land on one 64-byte line, and the hardware treats every update as a write to a shared line. This is false sharing: cores fighting over a line they do not actually share.
The effect is brutal and invisible. Each core's harmless `count[me]++` broadcasts an invalidate that knocks out the other cores' copies of the line; the line bounces between every core's cache exactly as in our trace, except now for no logical reason at all. A loop that should scale perfectly across cores can instead get slower as you add them, because they spend their time shuffling one contended line. The fix is pure layout: pad or align each core's data onto its own cache line (the cache-friendly habit from the last rung's final guide), so private data is also physically private. Same answer, no contention.
What coherence does and does not promise
One honest boundary before the next guide. Coherence guarantees that for one memory location, all cores eventually agree on a single, up-to-date value — never a stale photocopy. What it does not by itself promise is the ordering of writes to different locations as seen across cores. That larger question — in what order one core's stores become visible to another, and whether you need a memory barrier to force an order — is the memory consistency model, a distinct layer that sits on top of coherence. Coherence is per-location truth; consistency is cross-location ordering. Confusing the two is a classic beginner trap, so keep them separate.
And keep the cost in view, because it never disappears. MESI makes private, unshared data essentially free (that Exclusive trick), but genuine sharing — a lock, a shared counter, a producer–consumer queue — always pays for every cross-core handoff. This is why throwing more cores at a problem rarely buys proportional speedup: coherence traffic and contention grow with sharing, a foreshadow of Amdahl's law two guides from now. The next guide builds the tools you actually use on top of all this — atomic operations, compare-and-swap, and the locks made from them — every one of which is, underneath, a carefully choreographed dance through these very MESI states.