Waiting on Many Things: select and poll

One process, many channels, one problem

By the end of this rung you have collected a drawer full of channels. An anonymous pipe to a child, a couple of named FIFOs, maybe a Unix-domain socket to a peer, and your own standard input from the keyboard. Every one of them, thanks to everything being a file, is just a file descriptor you read() and write(). That uniformity has been a quiet blessing all along — but now it bites. Imagine a tiny chat hub: one process holding three connections, and a message could arrive on any of them at any time, in an order nobody can predict. You want to handle whichever speaks first. How?

The obvious move fails. If you call read() on the first descriptor, that call blocks — your process freezes inside the kernel until bytes show up on descriptor number one. Meanwhile the second connection could be shouting its head off and you would never notice, because you are asleep waiting on the wrong one. This is the blocking behavior you met in the files rung, now turned into a trap: a blocking read() commits your whole single-threaded attention to exactly one descriptor, and the world has many. You cannot read() three things at once.

Two bad escapes tempt you here, and naming them sharpens what the real fix must be. You could spawn one thread per descriptor, each blocked on its own read() — workable, but you now carry the full weight of threads, shared mutable state, and the locking from the previous rung, just to wait. Or you could set every descriptor non-blocking and loop forever, poking read() on each in turn; that never sleeps, so it pins a CPU core at 100% burning cycles to ask "anything yet? anything yet?". Both work and both are wrong. What you actually want is to hand the kernel your whole list and sleep until it wakes you the instant any of them is ready — paying nothing while idle.

Multiplexing: one call, many descriptors

The kernel offers exactly that deal, and it has a name: I/O multiplexing. The idea behind I/O multiplexing is to invert the question. Instead of you picking one descriptor and asking "give me data from this one" (which blocks), you hand the kernel a set of descriptors and ask the gentler question "tell me which of these is ready — readable, writable — and let me sleep until at least one is". One single system call watches the whole set at once. When it returns, it tells you the subset that can be acted on right now, and you read() only those, knowing each will return immediately without blocking.

Notice how cleanly this rests on something you have carried since the very first guide of this rung. Because a pipe, a FIFO, a socket, and the keyboard are all descriptors, a single select() call can wait on a wild mixture of them in one breath — a pipe from a child, a FIFO from a stranger, and a connected socket, side by side in the same set. The kernel does not care what is behind each number. That is everything is a file paying its biggest dividend yet: one uniform waiting primitive that works across every channel this rung introduced.

select(): the readiness loop, step by step

Let us make it concrete with the older of the two calls, select(). You build a descriptor set — a value of type fd_set, which you can picture as a bitmap, one bit per possible descriptor number. You clear it with FD_ZERO(), switch on the bit for each descriptor you care about with FD_SET(fd, &set), then hand the set to select() along with the highest descriptor number plus one. select() sleeps until at least one descriptor in the set is ready, then returns having modified your set in place: now only the ready descriptors have their bits left on. You walk your descriptors and test each with FD_ISSET(fd, &set) to see which ones survived.

FD_ZERO(&readset) to start with an empty set, then FD_SET(fd, &readset) for every descriptor you want to watch for incoming data. Track the largest fd you added; select() needs maxfd + 1 as its first argument.
Call select(maxfd + 1, &readset, NULL, NULL, NULL). The two NULLs skip the write-ready and error sets; the final NULL means wait forever (pass a timeout struct instead to wake after a deadline even if nothing is ready).
When select() returns a positive count, loop over your descriptors and test each with FD_ISSET(fd, &readset). For every fd whose bit is still set, call read(fd, ...) once — it is guaranteed not to block. Always check read()'s return: 0 means end-of-file, so close that fd and drop it from your set.
Crucially, rebuild the set before the next select(). Because select() overwrites the set with only the ready bits, you must FD_ZERO and FD_SET your live descriptors again every iteration — forgetting this is the classic select() bug where descriptors silently stop being watched.

poll(): the same idea, fewer sharp edges

select() carries some ancient baggage, and poll() is the cleaner sibling that fixes the worst of it while keeping the exact same readiness model. The first wart select() has is a hard ceiling: its fd_set bitmap has a fixed width, usually 1024 bits, so a descriptor numbered 1024 or higher simply does not fit — and writing past the bitmap is plain undefined behavior, not a clean error. The second is the rebuild chore you just saw: because select() clobbers the set, you re-list every descriptor on every loop. The third is the awkward maxfd + 1 argument. poll() drops all three.

Instead of a fixed bitmap, poll() takes an array you build of struct pollfd, one entry per descriptor. Each entry has three fields: fd (the descriptor), events (a bitmask of what you are waiting for, e.g. POLLIN for "readable"), and revents (which the kernel fills in with what actually happened). You pass the array, its length, and a timeout. The beauty is that poll() writes its answers into the separate revents field and leaves your fd and events fields untouched — so you do not rebuild the array each loop; you just zero out the revents (or ignore them) and call again. There is no 1024 limit either: the array is as long as you make it.

struct pollfd fds[2];
fds[0].fd = fd_a;  fds[0].events = POLLIN;   /* watch for readable */
fds[1].fd = fd_b;  fds[1].events = POLLIN;

for (;;) {
    int n = poll(fds, 2, -1);          /* -1 = wait forever; array unchanged */
    if (n == -1) { perror("poll"); break; }

    if (fds[0].revents & POLLIN) {      /* kernel reports fd_a readable */
        char buf[256];
        ssize_t r = read(fd_a, buf, sizeof buf);
        if (r <= 0) { close(fd_a); fds[0].fd = -1; }  /* -1 fd is ignored */
        else        { write(1, buf, r); }
    }
    if (fds[1].revents & POLLIN) {
        char buf[256];
        ssize_t r = read(fd_b, buf, sizeof buf);
        if (r <= 0) { close(fd_b); fds[1].fd = -1; }
        else        { write(1, buf, r); }
    }
}

The same two-descriptor wait with poll(). You set fd and events once; the kernel reports back through revents, so the array is reused without rebuilding. Setting an entry's fd to -1 tells poll() to skip it. Every call is checked.

The honest limit, and what comes after

Both select() and poll() share one structural flaw that you should know before you reach for them, and it is about scale. Each call hands the entire descriptor list across to the kernel, the kernel scans all of them to see which are ready, and on return you scan all of them again to find the ready ones. That is O(n) work in the number of descriptors, every single call. With ten or a hundred descriptors this is invisible. With tens of thousands of idle connections — a real web server holding 50,000 mostly-quiet sockets — you copy and scan 50,000 entries on every wakeup just to discover that two of them had data. The cost grows with the connections you are watching, not the few that are actually active, which is exactly backwards.

This is not a reason to avoid them — for a handful of descriptors, select() and poll() are perfect, portable, and in the standard. It is a reason to know their ceiling. The fix the industry settled on is a stateful interface where you register your descriptors with the kernel once, and it then hands back only the ready ones on each wait, with no rescanning of the idle multitude. That is Linux's epoll (epoll_create(), epoll_ctl(), epoll_wait()), with kqueue serving the same role on the BSDs and macOS. These are not in the C standard or POSIX and they are platform-specific, which is the tradeoff: you trade portability for the ability to scale to many thousands of connections cheaply.

Step all the way back and see what this rung built. You started not even able to send a byte across the wall between two processes; you end able to run one process that calmly juggles dozens of channels — pipes, FIFOs, sockets — sleeping when idle and waking the instant any one of them speaks. That single-threaded, event-driven shape, an idle process woken by readiness, is the heart of nearly every high-performance server and event loop you will ever meet, from nginx to Node.js. The mechanism underneath is always the same one you just built by hand: hand the kernel your list of descriptors, and let it tell you which are ready.