The device is just more addresses
By now you can register a driver, sort it into character, block, or network flavour, and field an interrupt when the hardware shouts. But we have danced around the most physical question of all: when the driver wants to tell the network card to start sending, or ask the temperature sensor for a reading, what does it actually do? There is no special "talk to hardware" instruction. The answer is beautifully blunt — the device's control knobs are exposed as memory addresses, and the driver reads and writes them like any other memory location.
This is memory-mapped I/O, MMIO. The chip exposes a block of device registers — each register is a small slot, often 32 bits wide, that means something specific: write 1 to this one to start a transfer, read that one to learn whether the device is busy, set a bit in a third to enable interrupts. The bus wires those slots onto physical addresses, say a window starting at 0xfebc0000. When the CPU does an ordinary store to 0xfebc0000, the bytes never reach RAM at all — they travel to the chip, and the chip reacts. MMIO is the reason a single uniform thing, the address space, can name both your data and the levers of the hardware.
There is one wrinkle that bites every beginner. The addresses the kernel works with are virtual, but the device window lives at a fixed physical address that no page table maps by default. So before touching a register, the driver calls ioremap(), handing it the physical base 0xfebc0000 and a length; the kernel installs a page mapping and hands back a virtual pointer you may dereference. Crucially, that mapping is marked uncached — we will see in a moment why a register absolutely must not be cached the way ordinary memory is.
Why volatile, and why uncached
A device register breaks an assumption the compiler holds sacred about ordinary memory: that if nobody in your code wrote to a location, its value cannot change, and that reading the same location twice in a row is wasteful and can be folded into one read. For a status register both assumptions are false — the hardware changes it under your feet, and you may need to spin reading it until a bit flips. Tell the compiler this with the volatile qualifier: it forces every access in the source to become a real load or store, none elided, none reordered with each other, none invented. ioremap() returns a volatile-qualified pointer for exactly this reason.
Caching is the deeper trap, and here is where the cache you met in the performance rung turns from friend to foe. If the register window were cached, a write to a control register might sit in the CPU's cache line for a while before reaching the chip — so the device would not start until who knows when — and a read might be served stale from cache instead of asking the live hardware. Both are catastrophic. That is why ioremap() maps the region uncached: every access goes straight to the device, in program order, no cache in between. This is also why you must never use plain memcpy() on MMIO; instead drivers use accessor helpers like readl() and writel(), which the kernel guarantees emit a single correctly-sized access to the device.
DMA: let the device do the carrying
MMIO is fine for poking a few control registers, but imagine receiving a 64 KiB network packet one 32-bit register read at a time: the CPU would burn millions of cycles just shuttling bytes, doing nothing else. So for bulk data we hand the job to the device itself. Direct memory access, DMA, is the hardware's ability to read and write main memory on its own, without the CPU copying each byte. The driver tells the device "here is a buffer at this address, this many bytes — go fill it," then steps away; the device's DMA engine streams the data straight into RAM and raises an interrupt only when the whole transfer is done.
This is gloriously efficient and quietly dangerous, because now two parties touch the same buffer. First, the DMA engine speaks physical addresses — it has no MMU doing virtual-to-physical translation for it — so the driver must obtain a physical ("bus") address for the buffer and program that into the device, never the virtual pointer the CPU uses. Second, the cache strikes again: if the CPU's caches hold a dirty copy of the buffer, the device may DMA stale data out, or write fresh data into RAM that a cached CPU read then ignores. So a DMA transfer is bracketed by cache synchronisation: flush before the device reads, invalidate before the CPU reads what the device wrote. The kernel's DMA API does this bookkeeping for you, but you must call it at the right moments — skip it and you get the corruption-minutes-later bug we keep warning about.
The IOMMU: a page table for devices
Letting a device write anywhere in physical memory is powerful and frightening at once. A buggy DMA descriptor — or a malicious one, on a device you do not fully trust — could splatter bytes across kernel data structures, since the device bypasses the CPU's MMU entirely. The fix is to give devices their own address translation. An IOMMU sits between the device and physical memory and does for DMA exactly what the MMU does for the CPU: it translates the addresses a device emits through a per-device page table, so the device sees a tidy I/O virtual address space and can only reach the pages the kernel explicitly mapped for it.
This buys two things at once. It is protection — a device confined by the IOMMU cannot corrupt memory it was never granted, which is what makes it safe to hand a real device straight through to a virtual machine. And it is convenience — because the IOMMU can scatter one contiguous I/O virtual range across many disjoint physical pages, even a vmalloc-style scattered buffer can be presented to a simple device as if it were one solid block. The same translation trick that protects you also papers over the physical-contiguity problem from the previous section. The cost, as always, is honesty: an IOMMU has its own translation cache that must be flushed when mappings change, and that flush is not free.
Putting one transfer together
Let us walk the life of a single receive, so the pieces lock into one picture. The phrasing "ring the doorbell" below is the everyday driver metaphor for the final MMIO write that tells the device a request is ready — it is just a writel() to a register, but naming it the doorbell makes the ordering rules vivid: the doorbell must ring last, after the buffer and descriptor are fully in place, or the device races off reading half-built data.
- Allocate a DMA-safe buffer (physically contiguous, via the kernel's DMA helper) and get back both a CPU pointer and a device-side bus address for the same memory.
- Build a descriptor in memory: write the buffer's bus address and length into the small struct the device's manual specifies, so the hardware knows where to put the data.
- Issue a write barrier so the descriptor's bytes are guaranteed visible to the device before the next step — the volatile-versus-barrier distinction made real.
- Ring the doorbell: a single writel() to the device's control register telling it to begin. The CPU now steps away and can do other work.
- The device DMAs the incoming data straight into your buffer, then raises an interrupt to announce completion.
- In the handler, synchronise the buffer for CPU access (invalidate any stale cache), and only now read the freshly arrived bytes — they are finally yours to use.
Notice how every theme of this rung converges in those six steps. MMIO rings the doorbell and reads status; DMA moves the bulk; the cache and barrier rules keep the two parties' views of memory consistent; the interrupt signals completion so we never busy-wait; and an IOMMU, if present, quietly confines the whole dance to authorised pages. The next and final guide in this rung steps back to weigh when even interrupts are too costly — the polling-versus-interrupt trade-off and NAPI — how the device tree tells the kernel which of these devices exist in the first place, and how the same care we just took with a DMA buffer applies when ferrying bytes across the wall to a user program.