a blocking versus non-blocking file descriptor (O_NONBLOCK)

Imagine asking a waiter for water. In blocking mode you stand frozen at the table, doing absolutely nothing else, until the water arrives. In non-blocking mode the waiter says 'not ready yet' and you immediately go back to your other tasks, checking again later. A file descriptor - the small integer your program uses to name an open file, socket, or pipe - can be set to either of these two behaviours, and the choice changes everything about how a server scales.

By default a descriptor is in blocking mode. When you call read(fd, ...) and no data has arrived yet, the kernel parks (suspends) your thread, gives the CPU to someone else, and only wakes you when bytes are ready or the connection closes. That is fine when one thread serves one connection. But set the O_NONBLOCK flag (with open() flags or with fcntl(fd, F_SETFL, ...)) and the call instead returns immediately: if data is ready you get it, and if not, read() returns -1 and sets errno to EAGAIN (the same value as EWOULDBLOCK on Linux), meaning 'nothing now, try again later.' A non-blocking write() that cannot accept all the bytes right now writes as many as it can and returns that shorter count, or returns EAGAIN if it can take none.

Why a blocking read does not scale: if each connection needs its own thread parked inside read() waiting, then ten thousand idle connections cost ten thousand parked threads, each with its own stack (often a megabyte or more) and scheduler overhead. Non-blocking descriptors are the foundation that lets ONE thread juggle thousands of connections: you ask each one 'any data?', skip the ones that say EAGAIN, and never freeze waiting on a single slow client. The catch is that you must now ask a readiness mechanism (select, poll, epoll, or kqueue) which descriptors are actually ready, or you will burn the CPU spinning in a pointless loop of EAGAIN.