The Virtual File System and the Buffer Cache

The problem: too many file systems, one set of calls

The first three guides in this rung built one file system from the ground up: a superblock describing the volume, an inode per file pointing at data blocks through direct and indirect blocks, a bitmap tracking free space. But look at any real computer and you do not see one file system — you see a crowd. Your laptop's main drive might be ext4 or APFS, the USB stick you just plugged in is FAT32, the disc is ISO 9660, and a folder might secretly be a network file system pulling bytes from a server across the room. They lay out inodes, directories, and free space in wildly different ways. And yet you open all of them with the same read(fd, buf, n) and write(fd, buf, n).

How can one set of system calls work over so many incompatible designs? The answer is the same trick the OS uses everywhere: put an abstraction layer in the middle. A program does not call ext4's code directly; it calls a generic file interface, and a layer underneath figures out which real file system is involved and forwards the request to that file system's own code. This is the virtual file system, or VFS, and it is the central idea of this guide. It is the thin waist of an hourglass: many programs above, many file systems below, and one narrow, common interface squeezing through the middle.

How VFS works: common objects, per-file-system code

The way VFS pulls this off is by defining a small set of generic in-memory objects that every file system must learn to speak. The most important is the vnode (or in Linux, the in-memory inode): a uniform, in-RAM stand-in for an open file, regardless of what file system it really lives on. Whether the file is on ext4, FAT32, or across the network, the rest of the kernel sees the same kind of vnode with the same fields and the same operations. The other generic objects describe a mounted volume, a directory entry, and an open-file state — a common vocabulary the upper kernel can use without knowing the dialect spoken below.

Here is the clever part. Each generic object carries a table of function pointers — a little menu of operations like read, write, lookup, and create — and each file system fills that menu in with its own code. When you call read() on a file, VFS does not contain the logic to read an ext4 file; it just follows the vnode's read pointer, which lands inside ext4's reader. This is plain object-oriented dispatch done with C structs: same interface, different implementation chosen at runtime by which pointers were installed. Adding a brand-new file system to the kernel becomes a matter of writing one new set of functions and registering them — the programs above never change.

A program calls read(fd, buf, n). The trap into the kernel turns fd into an entry in the open-file table, which points at the file's vnode.
VFS reads the vnode's generic read operation — a function pointer — instead of containing any file-system-specific logic itself.
That pointer lands inside the real file system's code (say ext4), which knows how to map the file's byte offset to a disk block using the inode and its indirect blocks.
Before going to the disk, that code asks the buffer / page cache for the block — the next section's whole job — and only on a miss does it actually issue a slow disk read.

The buffer cache: never touch the disk twice

Even with VFS routing requests perfectly, one brutal fact remains from the storage rung: the disk is achingly slow. A read from RAM costs tens of nanoseconds; a read from a spinning disk costs milliseconds once you add seek time and rotation — a gap of roughly a hundred thousand to one. If the OS went to the disk for every read() and write(), the machine would crawl. The fix is to keep recently used disk blocks in a region of main memory and check there first. That region is the buffer cache, and it is the single biggest reason your file system feels fast.

The mechanism is just caching, the same idea as the TLB you met in the paging rung — keep a fast copy of what you recently used. When the file system needs a block, it first checks the cache. A hit returns the block from RAM almost instantly. A miss means the OS reads it from disk, places a copy in the cache, and hands it over; the next request for that block is then a hit. Because real workloads reuse the same blocks constantly — the same directory, the same superblock, the same hot file — even a modest cache turns the overwhelming majority of accesses into RAM-speed hits.

Writes get the same treatment, but with a twist that matters. A write usually goes into the cache and returns to your program immediately, marking that cached block dirty (changed but not yet saved). The OS pushes dirty blocks to disk later, in the background — a policy called write-back. This is wonderful for speed: your program does not wait on the disk, and many small writes to the same block collapse into one. But it opens a dangerous gap, and you should feel the danger now: between the moment write() returns success and the moment the block actually reaches the disk, the data lives only in volatile RAM. A crash or power loss in that window loses it. Closing that gap safely is the entire subject of the next guide on crash consistency.

Read-ahead, and one cache to rule them

The cache helps most when the data you want is already in it. So the OS does something almost mischievous: it guesses your future. If it notices you reading a file block by block in order — the most common access pattern there is — it bets you will want the next blocks too, and fetches them before you ask. This is read-ahead (prefetching). When the bet pays off, your second, third, and fourth reads are instant hits on blocks the OS quietly pulled in while you were busy with the first. When it does not — say you seek randomly all over a file — read-ahead wastes a little bandwidth, so the OS watches your pattern and dials prefetching up for sequential streams and down for random ones.

There is one more unification worth knowing, because it ties this rung back to the memory rung. Modern systems do not keep two separate caches — one for file blocks and one for the pages of virtual memory. They merge them into a single page cache that holds file data and memory pages in the same pool of physical frames. This is what makes a memory-mapped file possible: when you map a file into your address space and read it like an array, a page fault simply pulls the file's page into the shared cache, and your normal memory loads read it directly — no read() call at all. File I/O and memory paging turn out to be the same machinery wearing two hats.

Putting the layers together

Step back and you can see the whole stack the file-system layers form, top to bottom. Your program sits at the top issuing read() and write(). The system-call layer catches those and hands them to VFS. VFS picks the right file system through the vnode's function pointers. That file system's logic translates a file offset into a disk block number using the inode and indirect blocks. Before any disk access, it consults the buffer / page cache. Only on a miss does the request fall to the block layer and the device driver, which may reorder pending requests with disk scheduling before the hardware finally moves the data. Each layer knows only its neighbors, and that strict layering is exactly what makes the giant system buildable.

  your program          read(fd, buf, n)
        |
  system-call layer     trap into the kernel; fd -> open-file table -> vnode
        |
  VFS                   follow vnode's read pointer (generic -> specific)
        |
  a real file system    inode + indirect blocks: file offset -> block #
        |
  buffer / page cache   HIT? return from RAM (fast, the common case)
        |  (miss)             MISS? read from disk, cache a copy
  block layer + driver  disk scheduling reorders requests, hardware moves data
        |
  the disk              ~ a hundred thousand times slower than RAM

One read() falling down the layers. Note that most reads stop at the cache and never reach the disk at all — that is the whole point.

This layered design is why a USB stick and a network share both appear as ordinary folders, why your editor can read a file without knowing its on-disk format, and why a single cache speeds up everything above it at once. It is also the platform the final guide builds on: once you accept that writes linger in a cache before reaching the disk, you are forced to ask what happens if the power dies mid-write — and the answers (journaling, copy-on-write, and log-structured designs) are how a file system survives a crash without turning your data into garbage. That is exactly where this rung ends.