training-data attribution

Training-data attribution asks the inverse of normal interpretability: not which input features drove this prediction, but which training examples did. Given a model behavior — a particular answer, a memorized phrase, a bias — TDA traces it back to the handful of training documents most responsible, so you can audit, debug, or credit the data. It is the practical umbrella over influence functions and several cheaper, more scalable cousins.

Methods trade fidelity for cost. The gold standard is empirical: retrain many models on random data subsets and regress each example's presence against the behavior (datamodels), expensive but faithful. Gradient-similarity methods (TracIn) attribute by accumulating dot products between the test gradient and each training point's gradient along the optimization trajectory, cheap and checkpoint-based. Random-projection methods (TRAK) linearize the model around its trained parameters and use Johnson–Lindenstrauss sketches of gradients to approximate the datamodel at a fraction of the cost. All produce an attribution score per training example for a given query.

TDA is increasingly central to copyright disputes, fact-tracing in language models, and detecting which data induced a failure. Its limits mirror influence functions: trajectory and linearization assumptions make scores approximate, and attributions can be unstable across training seeds.