memory ordering

You write your program top to bottom and naturally assume the machine runs it in that order, with every write you make instantly seen by every other thread. Both assumptions are wrong on modern hardware. For speed, the CPU and the compiler are allowed to reorder memory reads and writes, and each core may hold recent writes in its own buffers before they become visible to other cores. Memory ordering is the set of rules describing which reorderings are allowed and when one thread's writes are guaranteed to become visible to another. On a single thread none of this is observable; across threads it can produce baffling outcomes.

A concrete shock: two threads, x and y both start at 0. Thread 1 does x = 1; then reads y. Thread 2 does y = 1; then reads x. You might prove that at least one of them must read a 1. But on a real CPU with store buffering, both threads can read 0 — because each thread's write is still sitting in its core's buffer, not yet visible to the other, when the read happens. The program order said write-then-read, but the visible order was effectively read-then-write. This is not a hardware bug; it is the documented behavior, and the precise rules are spelled out by a memory consistency model.

Memory ordering is why naive synchronization code, including Peterson's solution and hand-rolled lock-free tricks, can be correct on paper yet fail on a multicore machine. It is also why you almost never reason about it directly in everyday code: properly written mutex locks, semaphores, and atomic operations include the necessary ordering guarantees (acquire/release semantics) so that, as long as you protect shared data with them, you see a sane, intuitive order. You only have to confront raw memory ordering when you step outside those tools and manipulate shared variables directly — which is exactly when concurrency bugs become nightmarish.