Shared Memory: The Fastest IPC

Why the other IPC tools are slower than they look

Everything you have built in this rung so far moves data by copying it through the kernel. When you write() into an anonymous pipe or a message queue, the bytes travel from your buffer, across the system call boundary into kernel memory, sit there, and then get copied a second time out to the reader's buffer when it read()s. Two copies and two trips into the kernel for every message. For a chat program or a shell pipeline that is invisibly cheap. For two processes shovelling a megabyte of pixels sixty times a second, those copies and syscall costs start to dominate.

Shared memory removes the middleman entirely. The idea sounds almost like cheating once you have the virtual-memory rung under your belt: instead of copying bytes between two private address spaces, the kernel arranges for one region of physical memory to appear in both processes' address spaces at once. After the setup, there is no copy and no syscall — process A stores a value at its pointer, and process B, reading through its own pointer, simply sees it, because both pointers ultimately translate to the same physical page. That is why shared memory is, byte for byte, the fastest IPC mechanism the operating system offers.

The one mechanism, two faces: mapping the same page twice

To see why this is possible at all, recall the virtual-memory rung. Each process has its own virtual address space, and the page table is what maps each virtual page to a physical one. Normally process A's page 0x4000 and process B's page 0x4000 point at different physical frames — that isolation is the whole point of virtual memory. Shared memory is what happens when you deliberately break the isolation in a controlled spot: you ask the kernel to point a page-table entry in A and a page-table entry in B at the same physical frame.

The addresses need not match. The shared region might sit at 0x7f00_1000 in A and 0x5500_a000 in B; what matters is that both virtual ranges translate, through their separate page tables, down to one shared set of physical frames. This is also why you must never store a raw pointer inside shared memory: an address that is valid in A is meaningless in B. If two processes need to link data structures inside the region, they store offsets from the start of the mapping, not absolute pointers. This is a classic, painful first bug, and the fix is a discipline, not a flag.

There are two POSIX paths to set this up, and they share the same final step. The older System V family uses shmget(), shmat(), and shmdt(). The modern, file-descriptor-friendly path uses shm_open() to create a named shared-memory object, ftruncate() to give it a size, and then mmap() to map it into your address space. Either way, the act that actually creates the shared mapping is mmap() (or shmat()), and it hands you back a pointer to the start of the region. From that pointer on, it is ordinary memory — you read and write it with plain C, no further syscalls.

A complete, error-checked example

Here is the modern POSIX recipe in one small program: create a named object, size it, map it, and write a counter into it. A second program (or a forked child) that opens the same name and maps it will read exactly what we wrote. Notice that every call that can fail is checked — shm_open(), ftruncate(), and mmap() each have their own failure value, and a program that ignores them will crash far from the real mistake. mmap() in particular does not return NULL on failure; it returns the special value MAP_FAILED, so you must compare against that, not against 0.

#include <sys/mman.h>   /* shm_open, mmap */
#include <sys/stat.h>   /* mode constants  */
#include <fcntl.h>      /* O_* flags       */
#include <unistd.h>     /* ftruncate, close */
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

#define SHM_NAME "/demo_counter"
#define SHM_SIZE 4096          /* one page is plenty */

int main(void) {
    /* 1. create (or open) the named shared object */
    int fd = shm_open(SHM_NAME, O_CREAT | O_RDWR, 0600);
    if (fd == -1) { perror("shm_open"); return 1; }

    /* 2. give the object a real size */
    if (ftruncate(fd, SHM_SIZE) == -1) { perror("ftruncate"); return 1; }

    /* 3. map it into our address space */
    void *base = mmap(NULL, SHM_SIZE, PROT_READ | PROT_WRITE,
                      MAP_SHARED, fd, 0);
    if (base == MAP_FAILED) { perror("mmap"); return 1; }

    close(fd);             /* the mapping survives the closed fd */

    /* 4. now it is just memory: write through the pointer */
    unsigned long *counter = base;
    *counter = 0;
    for (int i = 0; i < 5; i++) (*counter)++;
    printf("counter is now %lu\n", *counter);   /* prints 5 */

    munmap(base, SHM_SIZE);
    shm_unlink(SHM_NAME); /* remove the name when truly done */
    return 0;
}

/* build:  gcc -O2 -Wall main.c -lrt  &&  ./a.out */

The part shared memory does NOT do for you

Now the honest, hard half. The example above worked only because one process touched the counter. The instant two processes write the same region with no coordination, you are back in the exact failure from the synchronization rung: (*counter)++ is not one indivisible step but three — load, add, store — and two processes can interleave so that one increment is silently swallowed. This is a data race, and the operating system will not warn you. The bytes are shared; the turn-taking is not. Shared memory is raw shared mutable state, and raw shared mutable state is a bug waiting for a schedule that triggers it.

So shared memory is almost never used alone. It is paired with a synchronization primitive that both processes can see — most commonly a named semaphore (sem_open()) or a process-shared mutex placed inside the shared region itself. A semaphore works across the process boundary the same way it worked across threads: you wait before touching the data and post after, so only one process is in the critical section at a time. The pattern is identical to the threads case; only the scope widened from one address space to two.

There is one more honest gap that surprises people. A plain process-private mutex from the threads rung will not work in shared memory by default — a pthread mutex must be created with the PTHREAD_PROCESS_SHARED attribute set, or it is only valid within the one process that made it. And memory visibility has its own subtlety: a write by core 0 is not guaranteed instantly visible to core 1 without the right barriers. The good news is you almost never reason about that directly: a correctly used semaphore or process-shared mutex already includes the memory barriers, so if you lock around every access, ordering takes care of itself.

Putting it together, and where this fits

Here is the full life of a typical shared-memory channel, start to finish, so the pieces lock into one picture.

1. One process creates the object: shm_open() with O_CREAT, then ftruncate() to set its size in bytes.
2. Each process maps it: mmap() with MAP_SHARED returns a pointer into its own address space (the addresses may differ between processes).
3. They agree on a layout: a struct at offset 0, say, holding a length and a buffer — using offsets, never absolute pointers, inside the region.
4. Every read or write of the shared data is wrapped in sem_wait()/sem_post() (or a process-shared mutex lock/unlock), so only one process is inside at a time.
5. On shutdown each process calls munmap(); the last one to truly finish calls shm_unlink() to remove the name, or the object lingers until reboot.

Step back and weigh the trade. Shared memory is the fastest tool in the IPC toolbox because, after setup, the kernel is entirely out of the data path — no copy, no syscall per message. You pay for that with the two responsibilities the kernel used to handle for you: synchronization (you supply the semaphore or shared mutex) and layout discipline (offsets, not pointers; agreed structure). Reach for it when raw throughput matters and the processes are on the same machine — video frames, large datasets, high-frequency state. Reach for a pipe, a message queue, or a Unix-domain socket instead when you want the kernel to handle ordering, message boundaries, and flow control for you, and the copy is not your bottleneck.

One thread runs through everything you have met in this rung: a pipe, a FIFO, a signal, shared memory — each is a different answer to "how do two separate processes touch the same information?". Shared memory is the answer that trusts you the most and protects you the least. The final guide, Waiting on Many Things: select and poll, closes the rung by tackling the other half of real IPC programs — not how to move the data, but how to wait efficiently on several channels at once without burning a CPU spinning.