the shell pipeline

You have almost certainly typed something like cat log.txt | grep error | wc -l. The vertical bars chain small programs into an assembly line: the first command's output feeds the second's input, whose output feeds the third's. Each program does one job and passes its results along. This is a shell pipeline, and it is the single most visible everyday use of inter-process communication.

Here is what the shell actually does behind that one line. For each | it calls pipe() to make an anonymous pipe. Then it fork()s a child for each command. In each child, before exec()ing the real program, it uses dup2() to point the child's standard output (file descriptor 1) at the write end of one pipe and its standard input (file descriptor 0) at the read end of the previous pipe — this is redirection. So grep does not know it is in a pipeline at all; it just read()s from fd 0 and write()s to fd 1 exactly as if talking to a terminal. The kernel's pipe buffers carry the bytes between them. All the commands run concurrently, not one-finishes-then-the-next-starts; data streams through as it is produced.

This design is the heart of the Unix philosophy: build tiny tools that each do one thing well and read and write plain text streams, then compose them with pipes. The pipeline gives you concurrency, streaming (a 10 GB file flows through without ever fully fitting in memory), and natural backpressure (if wc is slow, grep's writes block, which slows cat) for free. The honest caveat: because pipes have no message boundaries, every tool in the chain must agree on a text format (usually lines), and an error in the middle is easy to miss — by default the shell reports only the last command's exit status unless you enable pipefail.