jemalloc, tcmalloc and mimalloc
/ JEM-alloc, TEE-see-malloc, MY-malloc /
Picture one popular food stall with a single cashier. When the lunch rush hits and a hundred people want to order at once, they all queue at that one cashier and wait. That is roughly what happens to a naive allocator when many threads call malloc() simultaneously: they all contend for one shared heap behind one lock, and threads stall waiting for each other. jemalloc, tcmalloc, and mimalloc are three industrial-strength replacement allocators designed primarily to remove that bottleneck so allocation scales across many cores.
They share a common architecture built from the ideas in this field. First, a thread cache (the tc in tcmalloc literally stands for thread-caching): each thread keeps a small private stash of free blocks per size class, so the overwhelmingly common case — a small malloc or free — touches only thread-local memory and needs no lock at all. Second, a central heap (or a set of arenas): when a thread's cache is empty it refills in a batch from a shared structure, and when its cache overflows it flushes a batch back; jemalloc in particular gives each thread one of several arenas to spread contention. Third, size classes and segregated storage organise everything, with a separate path for large allocations handed straight to the operating system. The differences are in emphasis: tcmalloc (Google) pioneered the per-thread cache plus central free list; jemalloc (originally Jason Evans, used heavily at Facebook/FreeBSD) emphasises multiple arenas and strong fragmentation control; mimalloc (Microsoft) uses sharded free lists and a clean free-list design for very low overhead.
Why this matters: on a multithreaded server, swapping the system malloc for one of these can dramatically cut lock contention and tail latency with no code change (you just link or preload it). The honest framing: they are not magic and not always better — for a single-threaded program the system allocator may be just as good, they use more memory in thread caches, and the right choice depends on your workload's allocation pattern. They are a worked example of the whole field's vocabulary — thread caches, central heap, size classes, arenas — assembled to solve the allocator-contention problem at scale.
# replace the system malloc at startup, no recompile: $ LD_PRELOAD=/usr/lib/libtcmalloc.so ./my_server # or link it directly: $ gcc -O2 main.c -ljemalloc -o my_server # the hot path: per-thread cache pop/push, NO lock taken
Drop-in replacements: each gives every thread a lock-free private cache so concurrent malloc scales across cores.
These allocators reduce contention, not work: a single-threaded workload may see little gain, and the thread caches trade extra memory for speed. Always measure on your real workload before switching.