Buffering, Caching, Spooling, and Async I/O

Why the kernel does more than pass bytes along

By now you can picture the full descent of an I/O request from the earlier guides in this rung: a program makes a call, it crosses into the kernel, a device driver pokes the controller's registers, and either DMA or an interrupt eventually signals that the bytes have moved. That machinery is enough to read one sector or send one keystroke. But it would be miserable to use raw. The disk delivers data in 4 KB blocks while your program wants a single byte; the printer accepts one page a minute while your program produces pages in a millisecond; the network hands you packets in unpredictable bursts. The hardware layer just moves bytes — it does nothing to bridge these mismatches.

So the kernel layers a set of I/O services on top of the raw transfer mechanism, sitting inside that device-independent layer you met in the previous guide. The four big ones are buffering, caching, spooling, and the choice of blocking versus non-blocking versus asynchronous I/O. Each one exists to solve a speed or a sharing mismatch, and the easiest way to understand them is to ask, for each, exactly which mismatch it fixes.

Buffering: a holding pen between two speeds

A buffer is just a region of memory that holds data in transit between a producer and a consumer that do not run in lockstep. Think of the IN tray on a desk: the mail carrier (the device) drops letters in whenever they arrive, and you (the program) pick them up whenever you are free; neither has to stand and wait for the other. Buffering solves three distinct problems at once, and it helps to keep them separate in your mind.

First, speed mismatch: a fast producer and a slow consumer (or the reverse) can each work at their own pace as long as the buffer absorbs the difference, exactly like the bounded buffer in the producer-consumer problem you saw under synchronization. Second, transfer-size mismatch: the disk insists on whole 4 KB blocks, but your program calls read for 50 bytes — the kernel reads the full block into a buffer once, then hands you bytes out of it on each call, so it does not re-read the disk fifty times. Third, copy semantics: when you write data, the kernel copies it into a kernel buffer immediately so your program can reuse its own memory at once, even though the bytes have not actually reached the disk yet.

There is an honest cost here worth naming. The copy semantics above mean that when read(fd, buf, n) returns, your bytes are in a kernel buffer, not necessarily on the platter. That is why a sudden power loss can lose recent writes, and why databases call fsync to force the buffer out to stable storage. Buffering buys smoothness, but it quietly inserts a window where data exists only in volatile memory — a real trade-off, not a free lunch.

Double buffering: never leaving the device idle

A single buffer has a frustrating gap. While your program is busy consuming the buffer's contents, the device must sit idle — it cannot refill the buffer you are still reading from, or it would overwrite data you have not consumed yet. So the device works, then waits while you work, then works again: the two take turns, and neither is ever fully busy. Double buffering fixes this with the oldest trick in the book — use two buffers and swap.

Picture two waiters and two trays. While you eat from tray A, the kitchen is already loading tray B. The instant you finish A, the waiters swap: you start on the full tray B, and the kitchen begins refilling the now-empty A. Neither the kitchen nor you ever stops to wait for the other. Applied to I/O: the device fills buffer B by DMA while the CPU processes buffer A; when both finish, they swap roles. If the device and the CPU take roughly the same time, double buffering can nearly DOUBLE throughput, because the two now overlap instead of alternating.

single buffer (serial):
  device fills B ----  CPU reads B ----  device fills B ----  CPU reads B ----
  (device idle while CPU works, CPU idle while device works)

double buffer (overlapped):
  device:  fill A | fill B | fill A | fill B | ...
  CPU:             read A | read B | read A | ...
  (the two now run at the same time, on different buffers)

Caching: remembering what you already paid for

A buffer holds data on its way somewhere; a cache holds a copy of data you might want again, so you can skip the slow trip to get it. The distinction is subtle but real: the same block of memory can play both roles, but the intent differs. A buffer exists for transfer; a cache exists for reuse. The kernel keeps a buffer cache of recently used disk blocks in RAM, betting on locality of reference — the same idea behind the working set and the TLB you met in the paging guides. Read a block once, and the next read of that block is answered from RAM in microseconds instead of from disk in milliseconds.

This is why running the same command twice often feels instant the second time, and why a freshly booted machine feels sluggish until its caches warm up. The cache is finite, so when it fills, the kernel must evict something — and it uses replacement policies that are close cousins of the page-replacement algorithms (an LRU-flavored choice, evicting what has gone longest unused). Be honest about a caching subtlety, though: a cache only helps if the data is actually reused, and a write must eventually reach the disk. A write-back cache delays that write for speed but risks loss on a crash; a write-through cache is safer but slower. Same trade-off as buffering, wearing a different hat.

Spooling: taking turns at a device that cannot be shared

Some devices simply cannot interleave work from several programs. A printer is the classic example: if two programs sent lines to it at the same time, your résumé and someone's grocery list would come out shuffled together on the same page. Spooling (the name comes from Simultaneous Peripheral Operation On-Line) solves this by never letting programs talk to the device directly at all. Instead, each program's output is written to a separate file in a spool directory, and a single dedicated daemon feeds those files to the device one complete job at a time.

It is exactly a takeout counter with a ticket system. Everyone places their whole order (writes a complete spool file) and walks away free; the single cook (the spooler daemon) works through the queue one order at a time, start to finish, so orders never get mixed. Notice what spooling really buys: it turns a non-shareable device into one that many programs can use without coordinating with each other, and it decouples the slow device from the fast programs — once your file is spooled, your program is done and can move on, even if forty jobs are queued ahead of yours.

Spooling and buffering are cousins but not twins, and beginners blur them. A buffer typically smooths ONE stream between one producer and one consumer. A spool serializes MANY independent jobs from many programs onto one device, holding each whole job (often on disk, not just in RAM) until its turn. The print queue, the mail queue, and a batch of jobs waiting for a tape drive are all spooling. The defining feature is whole-job serialization onto a device that cannot be shared mid-job.

Blocking, non-blocking, and asynchronous I/O

The last service is not about moving data but about how your program WAITS for it. There are three styles, and confusing them is one of the most common sources of mysterious program behavior. With blocking I/O — the default — when you call read and the data is not ready, the kernel puts your process to sleep, moving it off the CPU and onto a wait queue until the data arrives, then wakes it. Simple to reason about (the call returns only when done), but your thread can do nothing else meanwhile. This is the polite restaurant buzzer: you hand over your order and sit, doing nothing, until it lights up.

With non-blocking I/O, the call returns immediately no matter what: it gives you whatever data is ready right now, or a "would block, nothing yet" answer, and never puts you to sleep. Your program stays in control and can do other work, but you are now responsible for checking back. Done naively this becomes polling — asking "ready yet? ready yet?" in a loop, which is exactly the busy-waiting waste you learned to avoid. The grown-up version uses an event-notification interface (select, poll, epoll, kqueue): you hand the kernel a whole SET of file descriptors and block once until ANY of them is ready, then service just those. That is how a single thread serves thousands of network connections at once.

Asynchronous I/O goes further still. You issue the request and it returns immediately — but unlike non-blocking, the transfer genuinely proceeds in the background (the kernel and DMA carry it out), and the kernel notifies you LATER when the whole thing is actually done, via a signal, a callback, or a completion event. The key contrast: non-blocking says "is it ready for me to start?" and you still do the transfer yourself; asynchronous says "go do the entire transfer and tell me when it is finished." It is the buzzer that texts you the moment your food is bagged and ready — you went off and did other things, and the completion comes to you. One honest warning to retire a common misconception: none of this makes a single request faster — the disk still takes the same milliseconds. What changes is that your thread is free during those milliseconds, so the PROGRAM gets more done overall. As with everything in this guide, the win is overlap and throughput, not the speed of any one transfer, and it is not hardware parallelism — just one CPU cleverly refusing to waste time waiting.

1. Blocking read: call read(fd, buf, n); kernel sleeps your process on a wait queue; data arrives (interrupt fires); kernel wakes you; read returns with the bytes. You did nothing while waiting.
2. Non-blocking read: call read; it returns instantly with either some bytes or "would block"; you go do other work; later you ask again (ideally by blocking once on a whole SET via epoll). You drive the transfer.
3. Asynchronous read: submit the request; it returns instantly; the kernel performs the WHOLE transfer in the background; you do unrelated work; when it truly completes, the kernel delivers a completion event to you. The transfer drove itself.