NUMA-aware allocation
/ NOO-muh /
On a big multi-socket server, memory is not one uniform pool that every CPU reaches equally fast. Picture an office split across several floors, where each team has filing cabinets on its own floor. Grabbing a file from your own floor is quick; fetching one from another floor means a trip up the stairs. That is NUMA — Non-Uniform Memory Access: the machine has several memory nodes, each attached close to one group of CPUs, and a CPU reads its own node's memory faster than a remote node's. NUMA-aware allocation means the allocator tries to give each thread memory from the node nearest the CPU it runs on.
How it actually works hinges on a detail of the operating system: the standard local-node (or first-touch) policy. When you call malloc() and get an address, no physical memory is committed yet — the page is only really assigned to a physical node the first time some CPU touches (writes to) it, and the kernel places that page on the node local to the touching CPU. So the practical rule is: the thread that first writes to a page should be the thread that will mostly use it. A NUMA-aware allocator builds on this by keeping per-node arenas or per-thread caches pinned to the local node, so a thread's allocations come from, and are first-touched on, its own node. The payoff is real: a remote memory access can cost noticeably more latency and less bandwidth than a local one, so a server that accidentally scatters a thread's data across remote nodes can run much slower for no obvious reason.
Where this bites: high-core-count database and HPC workloads, where the classic mistake is one thread allocating and initializing a big array (first-touching every page onto its own node) and then many threads on other nodes hammering it remotely. The fix is to allocate-and-initialize in parallel so each thread first-touches its own slice, or to use NUMA-aware allocators and APIs (numactl, libnuma, mbind()) to control placement. The honest caveat: NUMA awareness only matters on multi-socket NUMA hardware; on a single-socket machine it is irrelevant, and getting it wrong (over-pinning) can hurt by starving a busy node while a neighbour sits idle.
# pin a process to node 0's CPUs AND node 0's memory: $ numactl --cpunodebind=0 --membind=0 ./db_server /* first-touch rule: the thread that FIRST writes a page gets it local. */ /* So initialize in parallel, not all from one thread: */ #pragma omp parallel for for (size_t i = 0; i < n; i++) big[i] = 0; /* each thread touches its slice */
Under first-touch, the thread that writes a page first decides its node; parallel init keeps each thread's data local.
malloc() returning an address does not place physical memory; under first-touch the page lands on whichever node first writes it. Allocating on one thread and using on another is the classic NUMA performance trap.