a conjugate prior
Updating a belief by Bayes' theorem usually means an ugly integral for the normalizing constant. A conjugate prior is a happy shortcut: a prior chosen so that, after multiplying by the likelihood, the posterior comes out in the SAME family as the prior — only its parameters change. You start with, say, a beta distribution; after the data, you still have a beta distribution; you just slide the parameters. The hard integral evaporates and updating becomes arithmetic.
Three classic pairs do most of the work. Beta-binomial: if your prior on a success probability p is Beta(a, b) and you observe s successes in n trials, the posterior is Beta(a + s, b + n - s) — you literally add successes to a and failures to b. Gamma-Poisson: if your prior on a Poisson rate lambda is Gamma(a, b) and you observe counts summing to S over n periods, the posterior is Gamma(a + S, b + n). Normal-normal: with a normal prior on an unknown mean and normal data of known variance, the posterior mean is a precision-weighted average of the prior mean and the sample mean, and the posterior is again normal. In each case the prior parameters behave like 'pseudo-data' you pretend to have already seen — a Beta(2, 2) prior is worth two prior successes and two prior failures.
Conjugacy is mathematically beautiful and pedagogically priceless, but be honest about its limits. It is a convenience, not a law: you should choose a prior because it reflects real belief, not merely because it is conjugate. Many realistic models have no conjugate prior at all, which is precisely the situation that forces you into Markov chain Monte Carlo. And conjugate updates assume the likelihood family is fixed and correct; a wrong model updates just as smoothly toward a wrong answer.
A new website button: start with prior Beta(1, 1), the flat 'I know nothing' prior. After 30 visitors, 6 of them click. The posterior is Beta(1 + 6, 1 + 24) = Beta(7, 25), whose mean is 7/32, about 0.22 — your updated estimate of the click rate, with the whole distribution telling you how unsure you still are.
Beta-binomial conjugacy turns Bayesian updating into 'add successes to a, failures to b'.
Conjugacy is a computational convenience, not a reason on its own to adopt a prior; a prior should encode real belief, and most realistic models have no conjugate prior at all.