Surviving a Crash: Journaling and Copy-on-Write

One file write is secretly three writes

By now you have built the whole machine. You know an inode holds a file's attributes and the addresses of its data blocks; you know a free-space bitmap tracks which blocks are taken; you know writes do not go straight to the platter but sit warm in the buffer cache first. Now we ask the question that has been lurking under all of it: what happens when the power dies in the middle? This is the subject of crash consistency, and it is the hardest, most humbling problem in file-system design.

Here is why it is so dangerous. Picture appending one block of data to a file. That single, innocent-looking action actually changes three different places on the disk: the new data block itself (the bytes you wrote), the bitmap (to mark that block as now used), and the inode (to record the new block's address and the file's new larger size). Three writes — and a disk can only truly commit one block at a time. A crash can strike in the gap between any two of them, freezing your file system half-finished.

And not all half-finishes are equal. Suppose the inode and the bitmap both got written, but the crash hit before the data block landed: your file now claims a block that contains stale garbage from whatever lived there before. Or worse, suppose the inode was updated to point at the new block but the bitmap was never marked: that block is reachable by your file yet still marked free, so the next file to allocate will grab the same block and two files will silently corrupt each other. A half-written file system is not just incomplete — it can be actively, dangerously inconsistent.

The old way: scan everything and hope

The first generation of file systems took a brute-force approach to the half-finished problem: do not prevent it, just clean it up afterwards. On every boot — or at least after an unclean shutdown — a repair tool (the classic Unix one is called fsck, "file-system check") walks the entire disk looking for inconsistencies. It rebuilds the free-space bitmap from scratch by following every inode's pointers, cross-checks that every reachable block is marked used and every used block is reachable, counts the links to each inode, and patches up whatever does not add up.

This works, but it has a fatal flaw that gets worse every year: time. fsck must examine the whole file system, so its cost scales with the size of the disk, not with how much was actually being written when the crash hit. A crash that touched one tiny file can still force a scan of a multi-terabyte volume — minutes, sometimes hours, of a server sitting dark and unavailable on every reboot. As disks grew from megabytes to terabytes, "scan everything and hope" went from acceptable to unbearable. We needed a way to recover that costs only as much as the work that was in flight.

Journaling: write your intentions down first

The dominant modern answer is the journaling file system, and its core idea is borrowed straight from how a careful person keeps a bank ledger: before you change anything for real, write down exactly what you are about to do, in a special reserved area called the journal (or write-ahead log). Only after the whole plan is safely recorded do you go and perform the real updates. This is write-ahead logging: the log entry always lands before the change it describes.

Why does this rescue us? Because it turns three scattered, individually-interruptible writes into one all-or-nothing transaction. The whole journal entry ends with a tiny commit record — a marker that means "this transaction is complete and final." The commit record is a single block, so it either gets written or it does not; there is no half. That one atomic block is the hinge the entire scheme swings on. Here is the full sequence for our three-write append.

1. Write the planned new versions of the data block, the bitmap, and the inode into the journal area — but do NOT touch their real homes yet. If a crash hits now, the real file system is untouched and perfectly fine; the half-written journal is simply ignored.
2. Once all those journal blocks are safely on disk, write the single commit record. This one atomic block flips the transaction from "tentative" to "final."
3. Now (and only now) perform the real updates: copy the data, bitmap, and inode from the journal to their actual locations. This step is called checkpointing.
4. After the real updates land, mark that journal transaction as free so its space can be reused for the next transaction. The journal is a small circular buffer, not an ever-growing record.

Recovery after a crash is now beautifully cheap. On reboot, the file system reads only the journal — a tiny, fixed-size region, not the whole disk. For each transaction it asks one question: is there a valid commit record? If yes, it simply re-does the real updates (this is safe to repeat, since copying the same blocks twice changes nothing — a property called idempotence). If there is no commit record, the transaction never finished, so it is thrown away as if it never happened. Either way the file system lands in a consistent state in milliseconds, no matter how big the disk.

Copy-on-write: never overwrite, always relocate

Journaling pays the double-write tax to make updates safe. A copy-on-write file system (such as ZFS or Btrfs) takes a radically different vow: never overwrite a live block in place — ever. When you change a block, write the new version to a fresh, free location, leaving the old block completely untouched. You met this exact trick in the memory rung as copy-on-write for forked process pages; here it becomes the organizing principle of the whole on-disk file system.

But if the data block moves to a new spot, the inode that points to it is now wrong — so the inode must be updated too. And in copy-on-write you cannot overwrite the inode either! So you write a new copy of the inode at a fresh location. That makes the directory pointing to the inode wrong, so that gets a new copy too. The change ripples upward, block by block, all the way to the very top of the tree — the root of the file system. Nothing below has been overwritten; you have grown an entire new path of fresh blocks alongside the old, still-valid tree.

  OLD tree (still 100% valid)        NEW blocks written alongside

        [root]  <--- old root           [root']  <--- new root
        /    \                           /     \
   [dir]    [dir B]   (shared,    [dir]      [dir B']
     |       unchanged) ------------^           |
  [inode]                                    [inode']
     |                                          |
  [data]                                     [data']  (the new bytes)

  the single atomic switch:  superblock's root pointer  [root] --> [root']

Here is the elegant payoff. The whole giant update is committed by flipping a single pointer at the top — usually a field in the superblock that says "the current root is here." That one write is atomic: either it points at the old root or the new one, never anything in between. Before the flip, a crash leaves you with the perfectly intact old tree. After the flip, the perfectly intact new tree. There is no half-state to repair and, in principle, no fsck and no separate journal needed at all — crash consistency falls out of the structure itself.

Pushing it to the limit: the log-structured file system

There is a third design that takes copy-on-write's "never overwrite, always relocate" idea to its logical extreme. The log-structured file system treats the entire disk as one enormous, append-only log. Every change — data, inodes, everything — is simply appended to the end of the log in one big sequential write; the disk head never jumps back to update something in place. The original motivation was speed, not safety: on a spinning disk, a long sequential write is dramatically faster than many scattered small writes, and the buffer cache absorbs more and more reads, so most disk traffic is writes.

Crash consistency comes along as a happy side effect, just as it did with copy-on-write: since you only ever append, the data already on the log is never disturbed. After a crash you find the last consistent point in the log and you are done. But the append-only design forces a hard problem into the open: the disk fills up with dead blocks — old versions of files that have since been rewritten further down the log. A background process called the cleaner (or segment cleaner) must continuously scavenge the log, finding segments full of mostly-dead blocks, copying the few still-live blocks forward, and freeing the segment. The cleaner is the price of the pure-log design, and tuning it well is famously delicate.

These ideas are not museum pieces. Plain journaling lives in the everyday ext4 file system and in Windows NTFS. Copy-on-write powers ZFS and Btrfs. And the log-structured idea, once a research curiosity for spinning disks, found a perfect second home inside flash storage: an SSD's controller cannot overwrite a flash page in place either, so its internal flash translation layer is itself a log-structured system in disguise. The crash-consistency strategies you just met are quietly running underneath nearly every device you own.

Choosing, and what crash consistency does not promise

Step back and the three strategies line up as one family with one shared insight: never let a crash catch you between two writes that must happen together. Journaling does it by writing your intent first and committing with one atomic record. Copy-on-write does it by building a whole new tree off to the side and committing with one atomic pointer flip. The log-structured design does it by only ever appending, so there is no in-place update to catch half-done. Each turns a dangerous multi-write into a single all-or-nothing switch — they just choose a different switch.

Now the honest limits, because every one of these mechanisms has a sharp edge. First, recall the big caveat: metadata journaling keeps your file system's structure consistent but can still hand you a file with a mix of old and new bytes — so if your data truly must be all-or-nothing, the application itself must use careful patterns (write to a temp file, then atomically rename). Second, all of this assumes the disk tells the truth about when a write is durable; a drive that lies about flushing its own cache can defeat even a perfect journal, which is why real systems insist on honest flush/barrier operations.

And a misconception worth killing outright: crash consistency is not the same as a backup. Journaling and copy-on-write protect you from a crash mid-write; they do nothing against a drive that physically dies, against you deleting the wrong file, against ransomware, or against fire. A consistent file system can still be a consistently empty one. The dirty bit you met in the cache only tells the OS a block needs writing back; it cannot conjure a second copy on another machine. Real durability is layered — crash consistency plus snapshots plus off-machine backups — and no single mechanism in this guide is the whole story.