JOVANA
Explore Library Glossary Getting Started Three Levels Fields How it works Mission
Join the mission
All guides

How a System Call Really Works

You already know a system call is the one guarded doorway from your program into the kernel. Now we open that door and walk through it, hinge by hinge: the wrapper, the ABI, the trap, the table, and the careful copy across the boundary that keeps a hostile program from steering the kernel into someone else's memory.

The same doorway, now from the inside

In the boot guide just before this one, we watched the firmware hand control to the kernel, and far back in the foundations rung we sketched a system call as the bank teller window: a user-mode program cannot reach into the vault, so it fills out a slip and the trusted kernel does the privileged work for it. That picture was true, but it was deliberately a black box. This whole guide pries the box open. By the end you should be able to trace a call like "read(fd, buf, n)" all the way from your source code, across the great divide into kernel mode, and back — and understand what guards every step.

Here is the single most useful reframing. From inside the kernel, a system call is not a function you call; it is an event that happens to the CPU. The processor was happily running your code in user mode, and then one special instruction yanked it across the boundary into a fixed kernel entry point — the same machinery that a hardware interrupt uses. So the whole mechanism is really the answer to one question: how does a program, which by design cannot run privileged instructions, deliberately and safely cause the CPU to start running the kernel's code instead of its own?

Keep one honest distinction sharp, the one you met as API versus system call. When you write "printf" or open a file in your language, you are calling a library function — an API in the loose sense. Most of that code is ordinary user-mode work: formatting strings, buffering bytes. It becomes a real system call only at the exact instant it crosses into the kernel, and a single library call may make zero, one, or many crossings. The API is the friendly face; the system call is the actual border crossing underneath it.

The wrapper and the ABI: agreeing on the slip

Between your friendly "read(fd, buf, n)" and the bare trap instruction sits a tiny piece of glue: the wrapper. It is a small stub, usually shipped in the C library, whose only job is to package your request the exact way the kernel expects and then fire the trap. Think of it as the standardized withdrawal slip at the bank: you cannot just shout your request across the lobby; you write it on the one form the teller will accept, in the boxes they read, in the order they read them.

The exact rules of that form are the system-call ABI — the application binary interface. The ABI is a strict, machine-level contract that nails down details a high-level programmer never thinks about: which CPU register holds the system-call number that says which call you want, which registers hold the arguments and in what order, where the return value comes back, and how an error is signaled. On 64-bit Linux, for instance, the convention puts the call number in one register and the first several arguments in a fixed set of others; the wrapper's job is simply to load those registers correctly. Get the contract wrong by even one register and the kernel reads garbage.

The trap: one instruction crosses the divide

With the registers loaded, the wrapper executes the trap instruction — on modern x86-64 this is the dedicated "syscall" instruction; older designs used a software interrupt. This single instruction is the actual border crossing, and it does several privileged things atomically, in hardware, so a program cannot stop halfway and end up in an undefined in-between state. It flips the mode bit from user mode to kernel mode, it switches to the kernel's own stack, and it jumps to one fixed, pre-arranged kernel entry point. Crucially, the program does not get to choose where in the kernel it lands — the destination was set by the kernel long ago, at boot.

Why must the entry point be fixed and chosen by the kernel? Because if a user program could trap to any address it liked, it could jump straight past the kernel's safety checks into the middle of a privileged routine — and the whole protection scheme would be worthless. The hardware enforces this with a table the kernel fills in at boot. For interrupts and traps on x86, that is the interrupt descriptor table: a small array, one entry per kind of event, each pointing at the trusted handler the kernel registered. The trap instruction indexes into that table, so the destination is always one the kernel pre-approved. It is the same family of mechanism whether the event is a keyboard interrupt, a page fault, or a deliberate system call.

There is a real cost to this crossing, and it is worth naming honestly. Switching modes, saving the user registers so they can be restored later, switching stacks, and draining the CPU's deep pipeline all take time — far more than an ordinary function call within your own program. This is exactly why the foundations guide warned that a program making millions of tiny system calls can be surprisingly slow, and why techniques like buffering exist: to do more work per crossing and so cross less often.

Inside the kernel: the table and the careful copy

Now control is inside the kernel at that one fixed entry point, in kernel mode, on the kernel's stack. The kernel's first move is to look at the system-call number the wrapper left in a register and use it as an index into the system-call table — a simple array where entry number N holds the address of the kernel function that implements call number N. Read is maybe number 0, write number 1, open number 2, and so on. Picture a switchboard with numbered jacks: the caller names a number, the operator plugs into exactly that jack. This indirection is what keeps the door narrow — there is a finite, fixed list of things a user program may ask for, and nothing outside the table is reachable.

Here is the part that separates a real kernel from a toy one: it does not trust a single thing the user program handed it. This is argument validation across the user/kernel boundary, and it is the beating heart of system-call implementation. Suppose the call is "read(fd, buf, n)". The kernel must check that fd actually names an open file you are allowed to read, that n is sane, and — most dangerously — that buf points into your own memory, not into the kernel's. A malicious program would love to pass a buf that points at kernel memory or at another process's memory, hoping the kernel will helpfully copy secret bytes there. So the kernel never just dereferences a user-supplied address; it uses special, carefully checked copy routines that verify the whole range belongs to the calling process before moving a single byte.

The full round trip, end to end

Let us put every piece together and trace one concrete call: your program runs "read(fd, buf, n)" to read n bytes from an open file into a buffer. Follow the journey across the divide and back, and notice how each step we built up — wrapper, ABI, trap, table, validation — appears exactly once, in order.

  1. Your code calls read(fd, buf, n). This enters the C library's wrapper, which is still ordinary user-mode code running in your process.
  2. The wrapper follows the ABI: it places read's system-call number in the agreed register and fd, buf, and n in the argument registers, then executes the trap (syscall) instruction.
  3. The hardware atomically flips the mode bit to kernel, switches to the kernel stack, and jumps to the one fixed entry point named by the descriptor table. Control is now inside the kernel.
  4. The kernel reads the system-call number, uses it to index the system-call table, and finds the address of the kernel function that implements read.
  5. That function validates everything: is fd an open file you may read, is n reasonable, and does the whole range buf to buf+n lie inside your own address space? If any check fails, it returns an error code instead of doing the work.
  6. If all checks pass, the kernel does the privileged work — it fetches the bytes from the file and uses the safe copy routine to move them into your buffer.
  7. The kernel places the result (the number of bytes read, or a negative error) into the return register, flips the mode bit back to user, and returns to the wrapper, which hands the value back to your code as if read had been an ordinary function all along.

Step back and admire the shape of it. A system call is a tightly choreographed dance in which the only way across the great divide is one narrow, kernel-chosen door, and the kernel treats every visitor as a stranger until proven otherwise. The wrapper hides the ugliness, the ABI pins down the exact handshake, the trap performs the crossing atomically, the table keeps the set of requests finite and fixed, and validation defends the boundary. This same skeleton — trap into a fixed entry, dispatch through a table, validate, do privileged work, return — is what you will recognize again in the next guide when a hardware interrupt, rather than your own program, is the thing that yanks the CPU into the kernel.