avoiding allocation on the hot path
Imagine a kitchen during the dinner rush. The hot path is the small set of steps that runs over and over for every order — and stopping mid-rush to drive to the store for a fresh pot each time would be absurd; you keep pots ready and reuse them. In code, the hot path is the inner loop or per-request routine that dominates run time, and calling the memory allocator there (malloc() / free(), or in managed languages, anything that creates garbage) is the programming equivalent of that store run.
Why allocation is costly on the hot path comes down to several effects. A call to malloc() is not free: it may walk free lists, take a lock (so threads contend), touch cold metadata, and occasionally ask the operating system for more memory — and the memory it returns is usually cold in cache, so the first access stalls. In garbage-collected languages there is a second, subtler cost: every allocation adds to the heap the collector must later scan, so a high allocation rate triggers more frequent or longer GC pauses, which show up as latency spikes precisely in your tail. The discipline, often called allocation-rate tuning, is to do the allocations OUTSIDE the hot loop and reuse the memory inside it: allocate buffers once up front and reuse them across iterations; keep a pool or free-list of reusable objects and return them rather than freeing; reserve a container's capacity before the loop so it never grows mid-loop; and prefer the stack or a small fixed buffer for short-lived scratch data. The goal is a hot path that, in steady state, allocates nothing at all.
It matters most for latency-sensitive and real-time code, where a single allocator hiccup or GC pause is the difference between meeting and missing a deadline, and for highly parallel code, where allocator lock contention silently caps scaling. The honest caveats: this is a targeted technique, not a blanket rule — allocating on a cold path that runs rarely is completely fine, and pre-allocating or pooling everywhere adds complexity, wastes memory, and can introduce its own bugs (reused buffers that still hold stale data, lifetimes that are now manual). Modern allocators with per-thread caches (jemalloc, tcmalloc) make a single allocation much cheaper than the folklore suggests, so measure your actual allocation rate and its effect on the tail before contorting code to avoid it.
// HOT PATH allocates every iteration -> malloc/free churn, cache-cold buffers for (...) { char *buf = malloc(4096); process(buf); free(buf); } // reuse one buffer allocated outside the loop -> zero allocation in steady state char *buf = malloc(4096); for (...) { process(buf); } free(buf);
Hoist the allocation out of the loop and reuse the buffer: the steady-state hot path then allocates nothing.
This is targeted, not universal: allocating on a rarely-run cold path is fine, and pooling everywhere wastes memory and risks stale-buffer bugs. Measure your allocation rate and its tail impact before reorganizing code to avoid malloc().