Significance Level (alpha)

The significance level, written α (alpha), is the threshold you set in advance for how small a p-value must be before you reject the null — and equivalently, the false-alarm rate you are willing to tolerate. The conventional choice is 0.05, meaning a 5% chance of crying “effect!” when nothing is there.

Calling a result “statistically significant” just means p < α; it is a yes/no decision, not a measure of importance. The 0.05 line is a social convention, not a law of nature, and chasing it (running tests until you cross it) inflates false positives — which is why pre-registering α and reporting effect sizes matter more than the label.