Where the bytes actually go
By now you can park one thread on a hundred thousand non-blocking sockets, wake only on real work, and either react to readiness or be handed completions. But step back and watch a single byte travel. A web server reading a file from disk and pushing it to a socket does something quietly wasteful: the kernel reads the file into the page cache (one copy, into kernel memory), then read() copies it from the page cache into your buffer in user space (a second copy), then write() copies it from your user buffer back into a kernel socket buffer (a third copy), from where the network card finally takes it. Three copies of the same bytes, and you never even looked at them.
Each of those copies costs memory bandwidth and CPU cycles, and each crossing of the user/kernel boundary is its own expense — recall from earlier that a system call is not a free function call: it traps into the kernel, switches mode, and on modern hardware pays a tax for mitigations against speculative-execution attacks. When you are pushing gigabytes per second to thousands of clients, copying data you never touch is the single most embarrassing line on a profiler's flame graph. The whole point of the two ideas in this section is to delete copies and syscalls you should not be paying for.
Zero-copy: let the kernel move it for you
Zero-copy I/O is the family of techniques that move data from one file descriptor to another without ever dragging it through user space. The classic Linux primitive is sendfile(): you say sendfile(out_sock, in_file, &offset, count) and the kernel splices the file's page-cache pages straight toward the socket. Your three copies collapse toward one — or even toward zero data copies, since the network card can fetch the page-cache pages directly by DMA. Crucially, the bytes never enter your address space at all, so the two boundary-crossing copies simply vanish.
The honest catch is that zero-copy only works when you genuinely do not need to look at the data. If your server has to gzip the file, encrypt it with TLS, or rewrite a header, the bytes must come into user space so your code can transform them — and sendfile() no longer applies. This is why a plain static-file server gets enormous wins from sendfile() while an application that touches every byte gets none from it. Be suspicious of any blog post that presents zero-copy as a universal speedup; it is a precise tool for the pass-it-through-untouched case, and modern TLS offload to the network card is exactly an attempt to win it back for encrypted traffic.
Vectored I/O: one syscall, many buffers
Now the syscall-count axis. Real messages are rarely one flat blob — a network packet might be a small fixed header, then a variable body, then a trailing checksum, each living in a different buffer in your program. The naive move is three separate write() calls, paying the boundary-crossing tax three times. Vectored I/O (also called scatter/gather I/O) collapses that into one. You build a small array of (pointer, length) pairs — on POSIX this is an array of struct iovec — and hand the whole array to writev() or readv() in a single syscall.
On the write side, writev() gathers: the kernel walks your iovec array and sends header, body, and checksum as one contiguous stream on the wire, with no need for you to first memcpy() them into one big staging buffer. On the read side, readv() scatters: the kernel takes one incoming chunk and spreads it across your separate buffers — say the first 16 bytes into a header struct and the rest into a body buffer — again in one call. You save both the extra syscalls and the staging copy, which is a small zero-copy win riding along for free.
/* send a 3-part message in ONE writev() instead of 3 writes */
struct iovec iov[3];
iov[0].iov_base = header; iov[0].iov_len = 16;
iov[1].iov_base = body; iov[1].iov_len = body_len;
iov[2].iov_base = crc; iov[2].iov_len = 4;
ssize_t n = writev(sock, iov, 3); /* one syscall, one stream */
if (n < 0) { perror("writev"); /* handle EAGAIN, EINTR, real errors */ }
/* PARTIAL WRITE WARNING: n may be < 16 + body_len + 4.
you must advance past the n bytes already sent and writev the rest. */The trap is one you already met: writev() can write fewer bytes than you asked, exactly like write(). When n comes back smaller than the total, the kernel has accepted a prefix of your gathered stream — maybe all of the header and half the body. You cannot just retry the same iovec array, or you would resend bytes the peer already got. You must walk forward by n bytes, trimming the iovec array to skip what was sent, and call writev() again on the remainder. Getting this resume logic right is the single most common vectored-I/O bug, and it is why production code wraps it in a small helper.
Scaling accept: one listening socket, many cores
Now zoom out from one connection to the moment connections are born. Every server has a single listening socket on its port, and new clients arrive there to be accept()ed. With one event-loop thread that is fine, but to use all your cores you want several threads — and now they must share that one listening descriptor. The instant a connection arrives, the kernel may wake every thread blocked in epoll on that socket; they all rush to call accept(), exactly one succeeds, and the rest get EAGAIN and go back to sleep having done pure waste. This is the thundering herd you filed away in guide 2, and at high accept rates it can dominate your CPU.
The modern, clean fix is SO_REUSEPORT. With this socket option, several independent listening sockets — one per worker thread or process — may all bind() to the same address and port. Instead of N threads fighting over one queue, the kernel keeps N separate accept queues and hashes each incoming connection to exactly one of them, by a tuple of the client and server addresses. Each worker calls accept() only on its own socket and is woken only for its connections. The herd is gone by construction: there is nothing to stampede toward, because each thread has a private door.
- In each worker, create its own listening socket() — do not share one descriptor across threads.
- Set SO_REUSEPORT on it with setsockopt() BEFORE bind(), and check the return value — without the option, the second bind() to the same port fails with EADDRINUSE.
- bind() each socket to the same address and port, then listen() on each — the kernel now owns one accept queue per socket.
- Register each worker's own listening socket in that worker's own epoll, and run an ordinary accept loop — each thread sees only the connections the kernel routed to it.
The ceiling, and what lies beyond it
Stack the wins and you reach an impressive ceiling: non-blocking sockets, epoll or io_uring watching them, zero-copy for the pass-through bytes, vectored I/O to batch the rest, and SO_REUSEPORT spreading accept across every core. This is the recipe behind servers that hold a million connections — the C10M target — on commodity hardware. But honesty demands naming what still limits you: every packet still passes through the kernel's network stack, and at tens of millions of packets per second even a perfectly tuned stack becomes the bottleneck, dominated by per-packet syscall and interrupt overhead.
Beyond that ceiling lies kernel-bypass networking — frameworks like DPDK and approaches like AF_XDP that let a user-space process talk to the network card almost directly, polling its queues without a syscall per packet. You trade away the kernel's protection, its ready-made TCP/IP stack, and a great deal of simplicity for raw packets-per-second; you often re-implement networking in user space and dedicate whole CPU cores to busy-polling. It is the right tool for routers, load balancers, and trading systems, and the wrong tool for almost everything else. Know that it exists and what it costs, so you can recognize the rare problem that actually needs it.
One last caution before you go: none of these techniques is something you reach for blind. Every one of them — zero-copy, vectored I/O, SO_REUSEPORT, kernel bypass — adds complexity and new failure modes, and pays off only when the bytes or the syscalls are genuinely your bottleneck. The rung's discipline holds to the end: measure first, find the real cost on a flame graph, and then apply the precise tool that deletes it. A static-file server craves sendfile(); a packet router craves DPDK; most services in between want neither and are better served by clean, correct, partial-write-aware code. You now have the whole map of high-performance I/O — from one blocking read() to a million connections — and, just as importantly, the judgment to know which rung of it you actually need.