the thundering-herd problem

Imagine one cookie placed on a table and twenty hungry children all told 'go when you see food.' At the cookie's appearance all twenty leap up, race over, and nineteen find nothing - having spent energy for nothing and trampled each other on the way. The thundering-herd problem is exactly this in software: many threads or processes are woken to handle a single event, but only one can actually take it, so the rest wake, contend, and go back to sleep, wasting CPU and causing scheduler churn.

The classic case is many workers all blocked waiting to accept() on the same listening socket. When one connection arrives, a naive kernel wakes ALL the waiters; they all race into accept(); one succeeds and the rest get EAGAIN or are put back to sleep. The same pattern bites when many threads all wait on the same epoll instance for the same listening socket - an incoming connection can wake them all. The fixes target 'wake only one': on Linux, EPOLLEXCLUSIVE on an epoll registration asks the kernel to wake just one waiter per event; classic servers like older Apache used an explicit accept-mutex so only one worker is allowed to sit in accept() at a time; and SO_REUSEPORT gives each worker its OWN listening socket so the kernel steers each new connection to exactly one of them, sidestepping the herd entirely.

Why it matters: at high connection rates a thundering herd turns what should be a quiet wakeup into a stampede of context switches and lock contention, capping throughput and adding latency precisely when load is highest. It is a recurring hazard in any design where multiple waiters share one wakeup source - accept loops, condition variables, and futexes all have their own versions and their own mitigations.