distributed tracing

In a small program you can follow a request by reading one stack trace. But a modern system handles one user click by passing it across a dozen separate services — a web front end calls an auth service, which calls a database, which calls a cache, all on different machines. When that click is slow, no single machine's logs tell you where the time went. Distributed tracing solves this by following ONE request end-to-end as it hops across services, stitching all the hops into a single coherent timeline.

Here is the mechanism in plain steps. The unit of work is a span: a single named operation with a start time, a duration, and some attributes (for example, the span 'charge-card' that took 790ms). A trace is the whole tree of spans for one request, all sharing a common trace id, with each span recording the id of its parent span — so you can reconstruct who called whom and how long each step took. The crucial trick is trace context propagation: when service A calls service B, it passes along the trace id and the current span id, usually in a request header (the W3C 'traceparent' header is the standard). Service B reads that header, makes its work a child span of A's span, and passes the context onward to whatever it calls. Done consistently, every service contributes its spans under the same trace id, and a tracing backend assembles them into one waterfall diagram showing exactly which service, and which call, dominated the request's latency.

It matters because in a many-service system distributed tracing is often the ONLY way to answer 'why was this specific request slow?' — it localises latency to the exact hop and surfaces problems that no per-service view can, like one slow downstream dependency dragging everything. The honest caveats: tracing only works if EVERY service propagates the context — one service that drops the header breaks the chain and you get a fragmented trace; capturing a span for every request is expensive at scale, so systems sample traces (keep a fraction), which means the specific trace you want may not have been kept unless sampling is biased toward errors and slow requests; and a trace tells you WHERE the time went across services, but you still often need the matching logs or a profile to learn WHY within the slow service.