The bit that says 'not here'
By the last guide you can read a page table in your sleep: a virtual address splits into a page number and an offset, the page number indexes the table, and the matching page-table entry hands back the physical frame number so address translation can finish. We waved past one little field in that entry. Every entry carries a valid bit — sometimes called the present bit — and it answers a blunt question: is this page actually sitting in physical memory right now, or not?
When the valid bit is 1, translation proceeds exactly as you learned. But when it is 0, the frame number in the entry is meaningless — there is no frame, because the page is not in memory. The hardware cannot fabricate the data out of nothing, so it does the only honest thing: it stops the instruction dead and raises a special exception. That exception is the page fault, and despite its alarming name it is not an error at all. It is a planned, routine event — the page table's way of saying 'this page exists in the program's world, but you'll have to go fetch it before I can give you a real address.'
Demand paging: load nothing until it's needed
Here is the bold idea that the valid bit makes possible. When you launch a program, the obvious plan is to copy the whole thing — code, data, the lot — from disk into frames before it runs. But most programs never touch most of their pages on a given run: error-handling code that never fires, a giant array you only half-fill, a help screen you never open. Copying all of that in is wasted work and wasted memory. So instead we copy in nothing, mark every page invalid, and start the program anyway. This is demand paging: a page is brought into memory only on demand — only when the program first reaches for it and faults.
Think of moving into a new house. The eager plan unpacks every box on day one. The lazy-but-clever plan leaves the boxes in the garage and unpacks a thing only the moment you first want it — the kettle when you fancy tea, a book when you reach for it. You end up unpacking far fewer boxes, and the ones you do are exactly the ones you use. Demand paging is precisely this laziness applied to memory, and it is why a multi-gigabyte application can start in a fraction of a second: it doesn't load the program, it loads the first handful of pages and lets the rest stream in as faults arrive.
What actually happens on a fault
A page fault is a delicate piece of hardware/OS choreography, because the faulting instruction was caught mid-flight and must later resume as if nothing happened. The hardware detects the invalid entry and traps; from there the operating system takes over and does the heavy lifting; then control returns to the very instruction that faulted, which now re-runs and sails through. Walk the steps once and the whole mechanism clicks into place.
- The CPU translates an address, finds the page-table entry's valid bit is 0, and raises a page-fault exception — abandoning the instruction before it commits any result.
- Control traps into the operating system's page-fault handler, which checks: is this a legal page the program is allowed to have, or a genuine illegal access (a wild pointer)? An illegal one is killed here — this is where 'segmentation fault' comes from.
- For a legal fault, the OS finds a free frame (evicting some other page if memory is full), and starts reading the missing page in from disk — a memory-mapped file, the program's executable, or the swap area.
- The disk read takes millions of cycles, so the OS hands the CPU to a different process meanwhile — the faulting program simply sleeps. When the data lands, the OS writes the real frame number into the entry and flips the valid bit to 1.
- The OS returns from the exception to the same instruction. This time translation finds a valid entry, the load or store completes normally, and the program never knows it paused.
Notice the division of labour, because it is the whole point. The hardware does the fast, common path: translate, check the valid bit, and on a hit produce a physical address in nanoseconds. The software — the OS — handles the rare, slow path: it decides which frame to evict, talks to the disk, and updates the table. This split is the soul of virtual memory: it is a hardware/OS co-design, where neither half could pull off the illusion alone.
The working set, and when paging turns ugly
Demand paging only works because of the same bet that powers caches: locality. Over any short window a program touches just a small handful of its pages — the loop it's running, the data it's chewing on. That handful is its working set. As long as the working set fits comfortably in physical memory, faults happen only at the start (to page each part in once) and then almost never. The program runs at full memory speed, and the cost of demand paging amortises to nearly nothing.
But push too far and the illusion shatters. If you run so many programs that their combined working sets no longer fit, the OS is forced to evict a page that some program is about to need again — which immediately faults, evicting another page that another program needs, which faults too. The machine spends all its time shuttling pages to and from disk and almost none running your code. This is thrashing, and it is virtual memory's worst failure mode: throughput collapses by orders of magnitude, the disk light stays on, and everything crawls. It is the honest reminder that paging buys a vast address space, not free RAM — overcommit memory badly enough and the cure becomes the disease.
Why this matters, and what's next
Step back and see how much one valid bit bought. Demand paging lets a program run with only its working set resident, lets the system run programs whose total size dwarfs physical memory, lets several processes share one read-only copy of a library page, and lets a giant file be treated as if it were memory. None of this is 'extra RAM' — it is the page table being honest about what's present and the OS quietly filling in the rest on demand. Virtual memory is address translation plus this lazy, fault-driven loading, working together.
But there is a cost we have politely ignored: every single translation, fault or no fault, needs the page table — which lives in memory. So an unhelped load would touch memory twice, once to read the table and once to read the data, doubling every memory access. That would wreck performance even when no page is ever missing. The next guide fixes exactly this with a tiny, fast cache of recent translations: the TLB, where a TLB miss is the everyday cost that, kept rare, is what makes the whole illusion fast enough to live with.