Toolformer (self-supervised tool use)

Toolformer is a method for teaching a language model to decide on its own when to call an external tool, which tool, and with what arguments — without hand-labeled examples of tool use. The clever twist is that the model's own sense of what is useful supplies the supervision: a tool call earns its place only if seeing the result makes the upcoming text easier to predict. The model effectively annotates its own training data with API calls.

The pipeline is sample, filter, fine-tune. First the model proposes candidate API calls at many positions in unlabeled text. Each call is executed and its result inserted; a call is kept only if it lowers the language-modeling loss on the following tokens by at least a margin compared with no call or with an empty result. The surviving calls produce an augmented dataset on which the model is fine-tuned with ordinary next-token prediction, so at inference it naturally emits calls where they help and reads back the returned values inline.

The self-supervised filter is elegant but its reach is bounded by the loss signal: tools whose benefit does not show up as lower next-token loss are hard to learn this way, and the approach assumes the answer-bearing result fits neatly into the surrounding text. It established the principle that tool use can be learned from utility rather than demonstrations, a foundation later agent training builds on.