continuous profiling

Traditional profiling is something you do once, on demand: you suspect a problem, you attach a profiler, you collect a profile, you look at it, you stop. But the slow request you most want to understand often happened at 3am last Tuesday under a load you cannot reproduce. Continuous profiling answers this by profiling ALL the time, on every machine, in production — keeping a low-overhead, always-on sampling profile running so that whenever you need to ask 'where was the CPU spending its time during that incident?', the data already exists.

Concretely, continuous profiling relies on a sampling profiler (not instrumentation), because sampling's overhead is small and roughly fixed, which is what makes 'always on' affordable. An agent on each host periodically (say a few dozen times a second) captures the call stack of running threads across the whole machine — often using the PMU or a timer, and increasingly using eBPF to do it cheaply and across all processes at once. These samples are continuously shipped to a central store, where they can be aggregated and rendered as flame graphs sliced by service, host, version, or time window. The result is that you can scroll back to any past minute and see what every CPU was doing, and you can diff two time windows (a differential flame graph) to see exactly what got more expensive after a deploy. The on-CPU view shows where compute went; a companion off-CPU view (built from blocked-time samples) shows where threads waited.

It matters because it converts profiling from a reactive, reproduce-it-first activity into an always-available record, which is the essence of observability applied to performance: the answer is captured before you knew the question. It also exposes fleet-wide waste — a function burning 3% of CPU across thousands of machines is an enormous bill that no single-run profile would highlight. The honest caveats: it is still SAMPLING, so it gives statistical proportions, not exact per-call timings, and it can miss very rare or very short events; storing continuous profiles for a whole fleet is itself a real cost in storage and bandwidth; and the low overhead is low, not zero — the act of profiling perturbs the system a little, which is the observer effect that the next term confronts head-on.