The cost of hiring a cook for every order
In the earlier guides you learned what a thread is — a flow of control inside a process, sharing the code and heap but keeping its own stack and registers — and you saw how a one-to-one model backs each user thread with a real kernel thread. That model is powerful, but it hides a bill. Creating a thread is not free: the kernel must allocate a stack, set up bookkeeping, and register the new thread with the scheduler. Tearing it down costs again. For a thread that lives for hours, this one-time cost vanishes into the noise. But for a thread that exists only to handle one quick web request and then die, the setup can cost more than the actual work.
Picture a busy restaurant that hires a brand-new cook the instant each order arrives, then fires them the moment the dish is plated. The hiring interview, the uniform, the walk to the kitchen — all of that, repeated for every single plate. By dinner rush the restaurant spends more effort on hiring and firing than on cooking. A server that spins up a new thread per incoming connection has exactly this problem, and under a flood of requests it can collapse: thousands of short-lived threads pile up, each demanding a stack and a scheduler slot, until the machine drowns in overhead rather than work.
The thread pool: a standing crew that never goes home
The fix is the same one a smart restaurant uses: keep a fixed crew of cooks on staff. They are hired once, in the morning, and they stay all day. When an order comes in, an idle cook grabs it; when the dish is done, that cook does not go home — they simply wait for the next order. This standing crew is a thread pool. The program creates a set number of threads up front, and those same threads are reused over and over for an endless stream of small tasks. The expensive hiring happens once, at startup; after that, handing a task to a waiting thread is cheap. The kernel never has to make a new thread mid-rush.
How do tasks reach the crew? Through a shared queue. Incoming jobs are placed onto a work queue, and the pool's threads pull jobs off it one at a time. This is exactly the producer-consumer problem you will meet head-on in the next rung: the part of the program accepting requests is the producer, dropping jobs into the queue; the worker threads are the consumers, taking jobs out. The queue is a piece of shared mutable state, so it must be protected — but that danger is the whole subject of the final guide in this rung, so hold the thought.
requests ---> [ work queue: J1 J2 J3 J4 ... ]
| | |
(consumers pull jobs off the queue)
v v v
+------+------+------+------+
thread pool -> | T1 | T2 | T3 | T4 | (fixed crew, reused)
+------+------+------+------+
each thread: loop { take a job; run it; go idle }When threads are too heavy: green threads and coroutines
A thread pool reuses threads, but each pool thread is still a full kernel thread, with a full-sized stack (often a megabyte or more) and a slot in the kernel scheduler. If you want not eight or eighty but a hundred thousand concurrent tasks — a hundred thousand open network connections, each mostly just waiting — even a pool is too heavy. The answer is to make the unit of work itself lighter. Enter lightweight threads: flows of control that the kernel never sees, scheduled entirely inside the program.
Green threads are exactly this: threads created and scheduled by a runtime or library in user space, with no kernel involvement. They are the user-level threads from the previous guide, taken seriously and made the default unit of concurrency. Because switching between two green threads is just a function call inside your own program — saving a few registers and jumping — it costs a tiny fraction of a kernel context switch, which must trap into the kernel. A green thread's stack can also start tiny and grow only as needed, so a million of them fit where a thousand kernel threads would not. This is why modern runtimes for highly concurrent servers lean on green threads.
A coroutine is the most explicit lightweight building block of all. An ordinary function runs from top to bottom and returns once. A coroutine can pause itself in the middle — yield control back to whoever called it — and later be resumed from exactly that point, with all its local variables intact. It is a function that knows how to bookmark its own place. This cooperative pause-and-resume is the engine under green threads and under the async/await style you may have seen: each task runs until it would have to wait, then yields so another task can run, all on a single thread.
Cooperative vs preemptive: who decides when to yield?
There is an honest catch with lightweight threads, and it is worth stating plainly. Kernel threads are scheduled preemptively: a timer interrupt can yank any thread off the CPU at any moment, so one greedy thread cannot freeze the others. Pure green threads and coroutines are usually scheduled cooperatively: a task keeps the CPU until it voluntarily yields. If one coroutine sits in a tight loop and never yields — or makes a blocking call that does not cooperate with the runtime — every other task on that thread is stuck behind it. The lightness is real, but so is this responsibility.
Good runtimes soften this. They schedule many green threads across a small pool of kernel threads — precisely the many-to-many model from the previous guide — so if one green thread blocks the kernel, the others can be moved to a different kernel thread and keep running. Some runtimes even insert yield points automatically to fake preemption. The takeaway is not that lightweight threads are worse, but that they shift work from the kernel into a runtime, and the runtime must be written with care.
Choosing your tool, and what every choice shares
So you have a small ladder of options, each heavier than the last. A bare pthread (or any one-to-one kernel thread) gives you true preemptive scheduling and real parallelism, but each one is costly, so you want only a modest number. A thread pool reuses a fixed crew of those kernel threads to absorb a stream of short tasks without paying the creation cost again and again. Green threads and coroutines pack enormous numbers of cheap, cooperative tasks onto a few kernel threads, ideal when most tasks spend their time waiting rather than computing.
- Mostly waiting (lots of network or disk I/O, thousands of idle-ish connections)? Reach for green threads or coroutines — huge concurrency, tiny cost.
- A steady stream of short, independent CPU jobs? Use a thread pool sized near your core count, so you get parallelism without per-job creation overhead.
- A few long-lived, heavily parallel computations? Plain kernel threads (pthreads) are fine — the one-time cost is negligible over hours of work.
But here is the thread running through all three — and the bridge to the final guide. Pool worker, green thread, or pthread, every one of these flows of control shares the same heap and the same global data of its process. The work queue, a shared counter, a cached result — the moment two of them touch the same mutable state at the same time, you have stepped onto dangerous ground. That hazard has a name, the race condition, and learning to tame shared mutable state with locks and other tools is exactly where this rung ends and the next great chapter of operating systems begins.