torch.compile

PyTorch normally runs eagerly — each operation executes the moment its Python line does — which is wonderful for debugging but leaves performance on the table. torch.compile keeps that eager feel while transparently capturing the program into a graph and compiling it, so you get compiler-level fusion and scheduling from a one-line wrapper around your model.

It works in two main stages. TorchDynamo hooks into CPython's frame evaluation to trace Python bytecode into FX graphs, falling back to eager execution at unsupported constructs by inserting graph breaks, so it handles arbitrary Python without demanding a static, fully traceable program. The captured graphs go to a backend, by default TorchInductor, which lowers them and generates fused Triton kernels for GPU (and C++/OpenMP for CPU). Guards record the assumptions — shapes, types — under which a compiled graph stays valid; when a guard fails, it recompiles. AOTAutograd traces the backward pass too, so training is compiled, not just inference.

Frequent graph breaks or shape changes cause recompilation and erode the speedup, so the practical work is reducing breaks and stabilizing shapes (or using dynamic-shape mode). When it fits, it delivers much of a hand-fused stack's benefit with almost no code change — the headline promise of the PyTorch 2 design.