Multiple Comparisons

The multiple-comparisons problem is the inflation of false positives that happens when you run many tests at once: each test has its own chance of a fluke, so across dozens of tests at least one “significant” result is almost guaranteed even if nothing is real. Test 20 useless hypotheses at α = 0.05 and you expect about one false discovery.

Fixes adjust the threshold to control the overall error rate. The Bonferroni correction divides α by the number of tests (conservative but simple); methods that control the false discovery rate, like Benjamini–Hochberg, are less strict and popular in fields with thousands of tests. The deeper lesson: deciding what to test after peeking at the data (p-hacking) silently runs countless hidden comparisons.