a thread pool
Picture a taxi company. The wasteful way to serve riders would be to hire, train, and fire a brand-new driver for every single trip. The sensible way is to keep a fleet of drivers on standby; when a ride comes in, an idle driver takes it, and when the trip ends the driver returns to wait for the next one. A thread pool is that standby fleet for a program: a fixed group of worker threads created once, up front, that sit waiting to be handed tasks from a shared queue.
Concretely, at startup the program creates, say, eight worker threads and a queue. Each worker loops: take a task from the queue (or wait if the queue is empty), run it to completion, then loop back for the next one. When the program has work — handle this request, process that file — it does not create a thread; it just drops a task into the queue, and the next free worker picks it up. The number of workers is usually tuned to the machine (often near the number of CPU cores) so the system stays busy without being overwhelmed.
The reason this beats creating a thread per task is twofold. First, thread creation and destruction are not free — especially with kernel-level threads, each costs a trip into the kernel — so reusing a fixed set of threads amortizes that cost away. Second, a pool bounds how many threads run at once, which prevents a flood of tasks from spawning thousands of threads and thrashing the machine. The honest caveat: a fixed-size pool can become a bottleneck if its workers all block waiting on something (a slow database, say), since a blocked worker is a worker not doing useful work, and the queue then just grows.
A web server using a thread pool keeps, say, 16 workers ready. When 1,000 requests arrive at once, it does not spawn 1,000 threads — it queues the requests, and the 16 workers chew through them 16 at a time. The server stays fast and stable instead of collapsing under thousands of brand-new threads.
Pre-create workers once, reuse them forever — cheaper and bounded.
A fixed-size pool caps concurrency on purpose, which is usually good — but if every worker blocks on something slow, the pool stalls and the task queue just backs up. Size and blocking behavior must be considered together.