The cache line is the real unit of memory
Everything in this rung has pushed one idea: the machine is not the simple model in your head, and performance comes from respecting how it really works — what this rung has been calling mechanical sympathy. Nowhere does that bite harder than memory. Earlier on the ladder you learned to picture memory as a flat array of bytes you reach one at a time. The hardware does not work that way at all. The CPU never fetches a single byte from RAM; it fetches a whole cache line — on essentially every modern x86 and ARM chip, 64 bytes — and parks that whole line in its cache. Touch one byte and you have pulled in its 63 neighbors for free.
This is why memory layout dominates so much performance. If the next byte you need is already in a line you pulled in, the access is nearly free — an L1 hit costs a few cycles. If it is not, you pay a cache miss that can cost hundreds of cycles while the CPU stalls waiting for RAM, idle, doing nothing. A loop that strides through memory in 64-byte-friendly order can run an order of magnitude faster than the identical loop hopping randomly, not because it does less arithmetic but because it wastes less of every line it pays to fetch. The line, not the byte, is the unit you must design around.
False sharing: a fight over a line nobody meant to share
Now put two threads on two cores, each updating its own counter. Logically they share nothing — thread A only ever writes counter[0], thread B only ever writes counter[1]. You would expect them to scale perfectly: two threads, two counters, no contention. But suppose counter is a plain `long counter[2]`, so counter[0] and counter[1] sit 8 bytes apart, both inside the same 64-byte cache line. Now every time A writes its counter, coherence invalidates B's copy of the line; every time B writes, it invalidates A's. The two cores ping-pong exclusive ownership of one line back and forth, thousands of times a second, each waiting on the other — even though their data never overlaps. This is false sharing.
The cruelty of false sharing is that the code looks perfectly correct and perfectly parallel. There is no race condition, no lock, no shared variable in the source — the threads genuinely touch different addresses. The contention is invisible at the level of C; it lives entirely in the hardware's 64-byte granularity. The symptom is maddening: you add a second thread expecting a 2x speedup and instead the program gets slower than single-threaded, because the coherence traffic costs more than the work. This is precisely the kind of scaling failure the Amdahl-and-Gustafson guide warned about — a hidden serial bottleneck — except here the bottleneck is not in your code at all, it is in the cache line your two innocent variables happen to share.
The fix is to force the two pieces of data onto different lines, so a write by one thread never invalidates the other's. You pad or align so each thread's hot data owns a full 64-byte line to itself. In C that is what over-alignment is for: declare the per-thread struct aligned to a cache line and let the rest of the line go unused. Yes, you spend memory — 56 wasted bytes per counter — to buy back the scaling you lost. It is one of the purest trades in this whole rung: a little space for the difference between negative and linear speedup.
// FALSE SHARING: both counters share one 64-byte line
long counter[2]; // counter[0], counter[1] are 8 bytes apart
// thread 0: counter[0]++ thread 1: counter[1]++
// -> coherence ping-pongs one line; 2 threads can be SLOWER than 1
// FIXED: each counter gets its own line (C11 / C23 alignment)
#include <stdalign.h>
struct alignas(64) padded { long value; }; // 64-byte aligned + sized
struct padded counter[2]; // now on separate lines
// thread 0: counter[0].value++ thread 1: counter[1].value++
// -> no cross-invalidation; scales as you expectedData-oriented design: lay it out for the machine
False sharing is one instance of a larger principle. The way you arrange data in memory is often the single biggest lever on speed, and the layout that feels natural to a programmer is frequently the worst one for the machine. The natural instinct is to bundle everything about one thing into one struct — position, velocity, color, name, health, all together — and keep an array of those structs. This is the array of structs layout, AoS. It reads beautifully. It is also a cache disaster for any loop that touches only one field.
Suppose you have a million game entities and a loop that only advances each one's position. With AoS, the positions are scattered every 64 bytes apart, one per struct, each surrounded by velocity and color and name that this loop never reads. Every cache line you fetch is mostly waste: you pull in 64 bytes to use the 8 you care about. The cure is data-oriented design: store one array per field — all positions contiguous, all velocities contiguous — the struct of arrays layout, SoA. Now your loop strides through a dense, packed array of positions, every cache line full of data you actually use, and the hardware prefetcher can see the linear pattern and fetch the next lines before you ask. This single transposition can be a several-fold speedup with no change to the arithmetic at all.
Allocation churn: the cost you forgot you were paying
The last hidden cost is the one you stopped seeing the day malloc() started working. Way back on the ladder you learned that malloc() and free() hand you memory from the heap, and you check their return values like a good citizen. What is easy to forget is that a call to malloc() is not free arithmetic — it walks free lists, may split or coalesce blocks, may take a lock to be thread-safe, and once in a while drops into the kernel to grow the heap. A single allocation is cheap. A million allocations a second, churning the same blocks in and out, is a tax on your throughput you may not even know you are paying. This is allocation churn, and allocating on the hot path is the way you accidentally sign up for it.
Allocation churn hurts in two compounding ways, and both connect back to everything above. First the direct cost: the bookkeeping inside the allocator, and the lock contention when many threads allocate at once — which on the wrong allocator becomes its own serial bottleneck straight out of the Amdahl guide. Second, and often worse, the cache cost: fresh allocations come back from wherever the allocator finds a free block, so a sequence of malloc() calls hands you addresses scattered across memory. Walk a list of freshly-malloc'd nodes and every step is a cache miss to a cold, far-flung line. The allocator did not just cost you its own time; it destroyed your locality, which is the very thing the last two sections were fighting to protect.
The cure is to stop allocating on the hot path. The two classic moves are: reuse — allocate a buffer once, outside the loop, and refill it each iteration instead of freeing and re-allocating — and a pool or arena allocator, which carves many fixed-size objects out of one big contiguous block you allocated once. A pool hands out objects in O(1) by bumping a pointer, frees them in a batch, and as a bonus keeps them packed together so iterating them is cache-friendly by construction. You are not writing a better general-purpose malloc(); you are exploiting a pattern malloc() cannot know about — that all these objects live and die together — to replace a thousand expensive scattered allocations with one cheap contiguous one.
Putting the whole rung together
Step back and the three topics of this guide are one idea wearing three hats. False sharing is two threads accidentally contending over a line; data-oriented design is laying out data so each line you fetch is full of work you need; allocation churn is scattering data across cold lines and paying the allocator over and over to do it. All three are about the same scarce resource — the cache line and the bandwidth to fill it — and all three are invisible in the source code and visible only when you measure. This is mechanical sympathy in its purest form: the bottleneck is not the instructions the CPU runs but the data it waits for.
And that closes the loop on the whole rung. You began by learning to measure, not guess and to read a flame graph. You learned to micro-benchmark without fooling yourself, to watch the tail and not just the average, and to know — through Amdahl and Gustafson — exactly how much speedup a parallel section can possibly buy. This guide gave you the next layer down: once profiling tells you a loop is memory-bound rather than compute-bound, these are the tools that fix it — pad away false sharing, reshape AoS into SoA, and kill allocation on the hot path. Faster code, almost always, is code that asks the memory system for less and wastes less of what it gets.