Signal, noise, and the detector
Every measurement is a wanted signal sitting on top of unwanted noise. The figure of merit is the signal-to-noise ratio (SNR). A particle detector at a collider records millions of collisions per second, almost all of them known background; any new physics hides in a tiny excess.
For a counting experiment dominated by a background of B events, the significance is the signal S measured in units of the background's Poisson fluctuation \sqrt{B}. This is the number that decides discoveries.
Why \sqrt{B}? Counting experiments obey Poisson statistics: if you expect B background events, they fluctuate by about \pm\sqrt{B} from run to run. A signal is only convincing when it exceeds that natural fluctuation by a wide margin. So the whole game is to collect enough data that S grows (linearly with luminosity) while the noise floor \sqrt{B} grows only as a square root.
The five-sigma standard
How large an excess counts as a discovery? Particle physics fixed a famously strict convention: a statistical significance of five standard deviations, Z=5\sigma. The p-value — the probability the background alone would fluke up an excess at least this big — is then vanishingly small.
A five-sigma upward fluctuation of a Gaussian happens about once in 3.5 million — the threshold physicists demand before claiming a discovery.
Why so strict? Because experimenters test thousands of histogram bins and channels — the look-elsewhere effect. With enough places to look, a 3\sigma bump (a 1-in-740 fluke) will appear somewhere by chance alone. Three sigma is called 'evidence'; five sigma is 'discovery.' The 2012 Higgs boson announcement waited for 5\sigma in the Standard Model's predicted mass window.
Worked example: is the bump a discovery?
Let us run the whole loop on a toy discovery, folding in everything from Guides 1-4: extract a signal, compute its significance, budget the data, and confront the systematics.
Significance grows only as the square root of the data, so reaching discovery from strong evidence still costs another ~56% of beam time — the 1/\sqrt{N} law, now budgeting an experiment.
Where this leads
The toy captures the real workflow. Simulate: Monte Carlo pseudo-experiments generate the expected background and its fluctuations. Measure: extract S with its error. Infer: compute Z, then fold in systematics. Modern experiments do this at staggering scale — the LHC's detectors and pipelines process petabytes, and increasingly the fits and classifiers are machine-learned.
The frontier of computational-experimental physics is widening fast: differentiable simulators that fit thousands of parameters by gradient descent; machine-learning surrogates that replace expensive Monte Carlo; lattice QCD and exascale astrophysics simulations; and 'digital twins' of entire detectors. But the discipline is unchanged from Guide 1 — a result is only as good as its honest uncertainty, and only nature, not the computer, has the final word.