kdump
/ KAY-dump /
A core dump captures a single crashed PROCESS. But what if the whole KERNEL crashes — a kernel panic that takes down the entire machine? You cannot ask the dead kernel to write a file about its own death, because the kernel that would do the writing is exactly the thing that just broke. kdump is the clever mechanism Linux uses to capture a memory image of the crashed kernel anyway, so the panic can be analysed afterwards.
Here is the trick, in plain steps. Long before any crash, at boot, the system reserves a small slice of physical memory and loads a SECOND, separate 'capture' kernel into it that normally sits idle. When the main kernel panics, instead of just halting, it uses a mechanism called kexec to jump directly into that already-loaded capture kernel without a full hardware reboot. Because the capture kernel lives in its own reserved memory, the crashed kernel's memory is left untouched and intact. The capture kernel then boots up minimally and writes out the old kernel's entire memory to disk (or over the network) as a file conventionally called vmcore. After the machine reboots normally, an engineer loads that vmcore into a kernel debugger (the crash utility, or gdb with the matching kernel symbols) and examines the panic: the kernel stack, the registers, what every CPU was doing, and the data structures, exactly as they were at the instant of the panic.
It matters because kernel panics are otherwise nearly impossible to diagnose — without kdump you get a brief message scrolling past on a console before the machine is gone, which is rarely enough to find a deep bug in a driver or the kernel itself. kdump turns a fatal, fleeting event into a durable artefact you can study at leisure. The honest caveats: it must be configured and the capture kernel and crashkernel memory reserved IN ADVANCE — you cannot decide to use kdump after a panic has already happened; the reserved memory is permanently unavailable to the running system; and analysing a vmcore is expert work requiring the precise debug symbols for that exact kernel build, so it is a serious tool for serious kernel and driver debugging, not an everyday convenience.
# reserved at boot via the kernel cmdline: crashkernel=256M # after a panic, the capture kernel writes /var/crash/.../vmcore $ crash vmlinux /var/crash/127.0.0.1-2026.../vmcore crash> bt # backtrace of the panicking task, as frozen at the panic
A pre-reserved capture kernel saves the panicked kernel's memory as a vmcore; afterwards the crash tool reconstructs the kernel state at the panic.
kdump must be set up before the crash: the capture kernel and its crashkernel memory are reserved at boot, so a machine without kdump configured gives you nothing but a console message when it panics.