data contamination detection

When a model is pretrained on a web crawl, the crawl often contains the very benchmark questions and answers you later use to evaluate it. The model can then score well by recall rather than by skill, and your leaderboard number is inflated. Contamination detection is the forensic effort to find out whether, and how much, benchmark data leaked into training — so reported capabilities can be trusted.

Detection methods fall into two camps. String-overlap methods search the (often inaccessible) pretraining corpus for n-gram matches against benchmark items, the same machinery as deduplication. Behavioral methods probe the model directly without corpus access: membership-inference-style tests compare the model's likelihood on canonical benchmark text versus paraphrased or shuffled variants (a model that memorized the original assigns it anomalously high probability), guided-prompting checks whether the model can complete a held-out continuation verbatim, and exchangeability tests check whether example ordering within a benchmark is statistically detectable. Each yields evidence, not proof.

It matters because contamination is increasingly the default failure mode of LLM evaluation. The hard part is that absence of detected overlap does not prove cleanliness, paraphrased leakage evades string matching, and model owners rarely release the corpus needed for a definitive audit — so the field leans on canaries and freshly-collected, post-cutoff test sets.