Bayesian inference
Imagine you are a detective with a hunch and then you keep finding clues. Bayesian inference is the disciplined version of that: you start with what you already believe about some unknown quantity, you see data, and you update your belief. The whole subject treats an unknown — the bias of a coin, the true effect of a drug, tomorrow's demand — not as a single fixed number to be hunted down, but as a quantity you describe with a probability distribution that gets sharper as evidence arrives.
Mechanically it is one equation in three actors. The prior, written p(theta), encodes your belief about the unknown theta before seeing data. The likelihood, p(data given theta), says how plausible the observed data is for each candidate value of theta. Bayes' theorem multiplies them and rescales to a valid distribution: posterior is proportional to likelihood times prior, that is p(theta given data) = p(data given theta) times p(theta) divided by p(data). The denominator p(data) = integral of p(data given theta) times p(theta) d theta is just the normalizing constant that makes the posterior integrate to 1. The posterior is your updated belief — a full distribution, not a single guess — and you can read off its mean, its mode, or an interval that holds 95 percent of its mass.
The power and the controversy both live in the prior. Because you must state your starting belief, two honest people with different priors can reach different posteriors from the same data — Bayesians answer that as data accumulate the likelihood swamps any reasonable prior and the posteriors converge. The reward is that the output is directly interpretable: a 90 percent credible interval really is an interval you believe contains theta with probability 0.9, which is what most people wrongly think a frequentist confidence interval means. The price is that real posteriors are usually impossible to compute by hand, which is why simulation and Markov chain Monte Carlo exist.
You suspect a coin is fair but are not sure, so your prior on its heads-probability p is gently centered at 0.5. You flip it 10 times and get 8 heads. The likelihood pushes hard toward larger p, the prior pulls gently toward 0.5, and the posterior lands in between — perhaps centered near 0.7 but still wide, because 10 flips is little data. Flip 1000 times and get 800 heads, and the posterior tightens sharply around 0.8: the data has overwhelmed the prior.
Posterior is proportional to likelihood times prior; more data makes the likelihood dominate.
The posterior is a full distribution, not a single number; collapsing it to its mean throws away exactly the uncertainty Bayesian inference was built to carry.