Blocking, Non-Blocking, and the C10k Problem

What "blocking" actually does

Back in the file and socket rungs you wrote the most natural loop in the world: open a connection, call `read(fd, buf, n)`, get some bytes, do something with them, repeat. What you may not have looked at closely is what `read()` does when no bytes have arrived yet. By default a file descriptor is blocking: if you ask to read and the data is not there, the kernel does not return empty-handed. It puts your thread to sleep — off the run queue entirely — and only wakes it when bytes show up. From inside your program, `read()` looks like one slow function call; underneath, your thread spent that time not running at all.

This is a genuinely good design for one job at a time. The thread is not spinning, not burning CPU, not polling — it is parked, costing nothing, until the kernel has real work for it. The price is hidden but absolute: a blocking call can serve exactly one file descriptor. While your thread sleeps inside `read()` on connection A, it cannot notice that connection B just sent a full request and is waiting for an answer. The thread is the unit that blocks, and a sleeping thread is deaf to everything except the one fd it is parked on.

The obvious fix, and why it hits a wall

If one blocking thread can only watch one connection, the obvious answer is one thread per connection. Each thread does its own naive blocking loop, sleeps when its socket is idle, wakes when bytes arrive — and they all proceed in parallel. This works, and for hundreds of connections it works well; it is how a great many servers were written, and you already have the tools for it from the threads rung. For a chat server with fifty users it is the right call. The model only breaks when the number gets large.

Now make the number ten thousand. Each OS thread carries a real stack — commonly the default is around 8 MiB of virtual address space reserved, even if only a fraction is touched. Ten thousand of those is on the order of tens of gigabytes of reservation, and the resident cost plus kernel bookkeeping per thread is far from free. Worse than memory is scheduling: with ten thousand runnable-or-sleeping threads, the kernel scheduler spends real time just deciding who runs, and every time one wakes another sleeps there is a context switch — saving registers, swapping page-table state, evicting cache lines. The cost of threads was a footnote at fifty and is the dominant cost at ten thousand.

This wall has a name. In 1999 Dan Kegel collected the problem under the label the C10k problem: how do you handle ten thousand concurrent connections on one machine? The hardware of the day could easily push the bytes; the bottleneck was the programming model, specifically the assumption of one blocking thread per connection. The insight that broke the wall was to stop pairing threads with connections at all. You do not need ten thousand threads to watch ten thousand sockets — you need a way for one thread to watch many sockets without going to sleep on any single one of them. That is exactly what non-blocking I/O makes possible.

Flipping the switch: O_NONBLOCK and EAGAIN

The switch is a single bit of state on the file descriptor. You set the `O_NONBLOCK` flag (typically with `fcntl(fd, F_SETFL, O_NONBLOCK)`), and now the fd is non-blocking: it makes the same `read()` and `write()` calls behave differently when data is not ready. Instead of sleeping the thread, the kernel returns immediately. If there was nothing to read, `read()` returns -1 and sets errno to `EAGAIN` (the same value as `EWOULDBLOCK` on Linux) — which is not an error in the scary sense, it is the kernel politely saying "nothing right now, try again later." Your thread keeps running with the CPU still in its hands.

/* one non-blocking read attempt, checked honestly */
ssize_t n = read(fd, buf, sizeof buf);
if (n > 0) {
    /* got n bytes -- process them */
} else if (n == 0) {
    /* peer closed the connection (clean EOF) */
} else { /* n == -1 */
    if (errno == EAGAIN || errno == EWOULDBLOCK)
        ; /* not ready yet -- DO NOT treat as failure; move on */
    else if (errno == EINTR)
        ; /* interrupted by a signal -- retry the read */
    else
        perror("read"); /* a real error */
}

But notice we have only traded one problem for another. A non-blocking fd lets one thread touch many sockets without sleeping — but how should the thread find out which sockets are actually ready? The naive answer is to loop over all ten thousand fds calling `read()` on each, collecting bytes where they exist and `EAGAIN` everywhere else. That is busy polling, and it is a disaster: with mostly-idle connections you burn 100% of a CPU asking ten thousand sockets "anything? anything? anything?" and getting `EAGAIN` nearly every time. We removed the sleeping-thread cost only to add a spinning-thread cost. Non-blocking I/O is necessary but not sufficient; it is half of the answer.

The other half: ask the kernel which are ready

The missing half is to let the kernel do the watching, since the kernel is the one that actually knows when bytes land on a socket. This is I/O multiplexing: you hand the kernel a whole set of file descriptors and a single call that blocks until at least one of them is ready, then returns telling you which ones. Now one thread sleeps on the entire collection — not busy-spinning, genuinely parked — and wakes only when there is real work, anywhere in the set. The original select() and poll() system calls did exactly this, and they are the conceptual heart of every event-driven server.

Put the pieces together and the new shape appears. Make every socket non-blocking, register them all with one multiplexing call, block on that single call, and when it returns with a list of ready fds, loop over only those doing non-blocking reads and writes that are guaranteed not to sleep — then go back and block on the multiplexer again. That outer cycle is the event loop, the backbone of the next several guides. One thread, parked when idle, instantly responsive when any of thousands of connections has data: the C10k wall is gone, and we never needed ten thousand threads.

Readiness versus completion: the fork in the road

There is one more idea to plant before the rung opens up, because it quietly splits every high-performance I/O system into two families. The `select`/`poll`/`epoll` style above is the readiness model: the kernel tells you when an operation would not block — "this socket is now readable" — and then you perform the `read()` yourself. The kernel watches; you do the work. This is the model behind Linux's epoll and BSD's kqueue, and it is what guides 2 and 3 of this rung build on.

The other family is the completion model: instead of being told an operation would not block, you ask the kernel to carry out the whole operation — "read 4 KiB into this buffer for me" — and the kernel notifies you only when the bytes are already sitting in your buffer, done. You do not call `read()` at all; the kernel does, and reports the finished result. Windows' IOCP has worked this way for decades, and Linux's io_uring brought a modern completion interface to Linux. This readiness-versus-completion distinction is the axis along which guides 2 and 4 divide, so it is worth fixing the one-line difference now: readiness says you may now act; completion says I have already acted for you.

Hold the whole map in view as you climb. You now have the two halves that defeat C10k — non-blocking file descriptors so no single call freezes the thread, and multiplexing so one thread watches thousands without polling — and the fork that organizes the rest: readiness (epoll, kqueue) coming next, completion (io_uring, IOCP) two guides on. Everything ahead — edge versus level triggering, the reactor and proactor patterns, zero-copy and vectored I/O, and scaling the accept loop toward C10M — is built on exactly these foundations. Keep the picture of one parked thread, woken only by real work, and you will not lose your bearings.