Peeking

Repeatedly checking an A/B test's results before it has collected its planned sample, and stopping the moment it looks “significant”. It feels responsible but it badly inflates your false-positive rate.

Here is the trap: results bounce around early on, so if you keep looking and stop at the first lucky spike, you will declare winners that are pure noise — your true error rate can balloon from 5% to 20% or more. The fixes are to decide the duration in advance and only judge at the end, or to use a method designed for continuous monitoring such as sequential testing.