Guessing a number: the point estimate
Imagine you run a small online shop. You want to know one number: out of everyone who will ever visit your site, what fraction actually buys something? That fraction is your true conversion rate. The problem is obvious — you cannot wait for every future visitor, and you certainly cannot ask the ones who have not arrived yet. The true rate is a fixed but unknown number sitting out in the world.
So you do the practical thing: you look at a sample. Last week 2,000 people visited and 60 of them bought something. Your best single-number guess for the true rate is simply 60 divided by 2,000, which is 0.03, or 3%. That single number — your one best guess for the unknown quantity — is called a point estimate.
It helps to separate two ideas that beginners often blur. The recipe — “count the buyers, divide by the visitors” — is the estimator: a fixed rule you could apply to any sample. The number that recipe spits out for this particular sample — 3% — is the estimate. Statisticians give the unknown true quantity a name (here, the population conversion rate, written p) and the estimate its own symbol with a little hat on top.
The point estimate of a proportion: number of “yeses” (x) divided by sample size (n). The hat on p-hat is the universal mark for “estimated from data.”
This is the whole job of statistical inference: reach from a sample you can see to a claim about the population you cannot. The point estimate is the first half of that job. The uncertainty around it is the second — and the more important — half.
Every estimate has error: the standard error again
Here is the key mental move. Your 3% came from one random sample. If the universe handed you a fresh, equally random week of 2,000 visitors, you would compute a slightly different number. Imagine repeating that thousands of times: thousands of weeks, thousands of point estimates, each a little different. Collect them all and they form a distribution — the sampling distribution of your estimate.
That picture is the heart of inference, and you met it in the previous guide. The estimates cluster around the true value, and thanks to the central limit theorem the cluster is bell-shaped — a normal distribution — even when the raw data (a yes/no purchase) is anything but. What we now want is one number that measures how wide that cluster is: how much our estimate bounces around from sample to sample.
That number is the standard error. It is simply the standard deviation of the sampling distribution — the typical distance between an estimate and the true value. Be careful: the standard error is not the spread of your raw data; it is the spread of the estimate itself. For a proportion it has a tidy formula, and for an average it has an even tidier one.
Left: the standard error of an average shrinks with the data’s spread (sigma) over the square root of the sample size. Right: the matching formula for a proportion. Both shrink like one-over-root-n.
Put your numbers in. With p-hat = 0.03 and n = 2,000, the standard error is the square root of (0.03 times 0.97 divided by 2,000), which works out to about 0.0038 — about 0.38 percentage points. So a typical week-to-week wobble in your estimate is under half a percentage point. That single number is what turns a naked guess into an honest one.
A large box labelled population, an arrow to a small box labelled sample, and an arrow to a bell-shaped curve of many sample estimates; the half-width of the bell is marked as the standard error.
Building a confidence interval, step by step
A point estimate is a dot on a number line. A confidence interval is a range drawn around that dot that says, in effect, “the true value is plausibly somewhere in here.” You build it by starting at your estimate and reaching out a certain number of standard errors on each side. That is the whole idea; everything else is bookkeeping.
- Compute the point estimate. Here, p-hat = 3%.
- Compute the standard error. Here, SE = 0.38 percentage points.
- Choose a confidence level. 95% is the customary default; 90% and 99% are also common.
- Look up the multiplier for that level. Because the sampling distribution is roughly normal, 95% corresponds to about 1.96.
- Multiply: margin of error = multiplier × SE = 1.96 × 0.38 ≈ 0.75 percentage points.
- Reach out both ways: interval = estimate ± margin = 3% ± 0.75% = [2.25%, 3.75%].
The confidence-interval recipe in symbols: take the estimate, then add and subtract the multiplier z-star times the standard error. The plus-or-minus part has its own name.
That plus-or-minus part — here 0.75 percentage points — is the margin of error. It is the same quantity a news poll means when it says “52% support, plus or minus 3 points.” The margin bundles together the standard error and your chosen confidence level into one easy-to-read radius around the estimate.
What “95% confident” actually means (and what it doesn’t)
Now the single most misunderstood sentence in all of statistics. You computed [2.25%, 3.75%]. The tempting reading is: “There is a 95% probability that the true conversion rate lies between 2.25% and 3.75%.” It sounds perfectly reasonable. Under the standard (so-called frequentist) view, it is wrong.
Why? Because the true rate is a fixed number. It is whatever it is — say 3.1% — and it does not move. Your interval, once computed, is also fixed. So the true value is either inside [2.25%, 3.75%] or it is not; there is no “95% chance” about a fact that is already settled, the way there is no 95% chance that 7 is greater than 5. The randomness was never in the truth. It was in your sample.
So what is the 95% about? It is a property of the procedure, not of this one interval. Picture repeating the entire exercise — draw a fresh random sample, compute a fresh interval — over and over. About 95% of those intervals would cover the true value, and about 5% would miss it entirely. “95% confident” is shorthand for “this interval came out of a method that is right 95% of the time.” This particular interval is one draw from a long-run-reliable machine; you just cannot know whether today’s draw is one of the lucky 95 or the unlucky 5.
A vertical line marks the true conversion rate. Dozens of horizontal confidence intervals from repeated samples stack up the page; most cross the line, while a highlighted handful fall entirely to one side and miss it.
If you genuinely want a statement like “there is a 95% probability the rate is in this range,” that statement does exist — but it requires a different philosophy. The Bayesian credible interval treats the unknown rate as itself having a probability distribution, and so it can legitimately be read as a probability about the value. That is a different machine with different inputs (you must supply a prior belief), and it gets its own guide on Bayesian inference.
Width: sample size, variability, and confidence
A wide interval is a vague answer; a narrow one is a precise answer. Three dials, and only three, control the width. Once you see them, you can read any interval — or design any study — with clear eyes.
Dial one is the sample size, n. More data narrows the interval — but remember the square-root law, so the width shrinks like one-over-root-n, not one-over-n. Concretely: to halve your margin from 0.75 points, you do not double the sample from 2,000 to 4,000; you quadruple it to 8,000, which drops the margin to about 0.37 points. The first thousand visitors buy you a lot of precision; the ten-thousandth buys almost none.
Dial two is the variability in the data itself — sigma for an average, or p(1−p) for a proportion. Noisier outcomes give wider intervals. For proportions this peaks at 50%, where outcomes are most unpredictable, and tightens as you approach 0% or 100%. You do not usually control this dial, but knowing it explains why some quantities are inherently harder to pin down than others.
Dial three is the confidence level itself. Demanding more confidence — 99% instead of 95% — pushes the multiplier up from 1.96 to 2.576 and widens the interval. There is no free lunch: you can only buy more certainty by accepting a vaguer answer. A 100%-confidence interval would be “between 0% and 100%” — perfectly safe and perfectly useless.
All three dials in one line: the confidence multiplier z-star (up = wider), the variability sigma (up = wider), and the sample size n under a square root (up = narrower, but with diminishing returns).
The same stacked confidence intervals, now with sliders for sample size and confidence level; increasing the sample size visibly shortens every bar, while raising the confidence level lengthens them all.
Confidence intervals beat bare p-values for decisions
A p-value (the star of the next guide) answers a yes/no question: is the effect distinguishable from zero? A confidence interval answers the question a decision-maker actually has: how big is the effect, and how sure are we? The interval contains the p-value’s answer and adds the magnitude the p-value throws away.
Make it concrete. Two A/B tests both come back “statistically significant, p < 0.05.” Test A’s 95% interval for the lift is [+0.1%, +0.3%]: real, but tiny. Test B’s is [+4%, +18%]: large, but very uncertain. The bare p-values cannot tell these two apart — both just say “not zero.” The intervals tell you everything you need: ship B-style wins, and ask whether A-style wins are even worth the engineering.
There is a neat correspondence: a 95% interval that excludes zero matches a test that is significant at the 5% level, and one that includes zero matches a non-significant test. But the interval shows something a bare “not significant” hides. “Not significant” can mean two opposite things: a tight interval hugging zero (good evidence the effect is small) or a huge interval straddling zero (we simply have no idea yet). Same verdict, completely different situations — and only the interval reveals which one you are in.
Worked example: estimating a conversion rate
Let us run the whole pipeline once, slowly, on fresh numbers. You launch a redesigned checkout page. Over the test window it draws 4,000 visitors and 220 of them convert. You want an honest estimate of the new page’s true conversion rate.
- Point estimate: p-hat = 220 / 4,000 = 0.055, i.e. 5.5%.
- Standard error: square root of (0.055 × 0.945 / 4,000) ≈ 0.0036, i.e. 0.36 percentage points.
- Margin of error at 95%: 1.96 × 0.0036 ≈ 0.0071, i.e. 0.71 percentage points.
- Interval: 5.5% ± 0.71% = [4.79%, 6.21%].
The finished interval. Read it aloud: our best guess is 5.5%, and we are 95% confident this method’s interval [4.8%, 6.2%] covers the true rate.
import numpy as np n, x = 4000, 220 p = x / n # point estimate = 0.055 se = np.sqrt(p * (1 - p) / n) # standard error = 0.0036 z = 1.96 # 95% multiplier (normal) lo, hi = p - z * se, p + z * se # 95% CI = (0.0479, 0.0621) print(round(p, 4), round(lo, 4), round(hi, 4)) # 0.055 0.0479 0.0621
Now decide. Suppose the old page converted at 6.0%. That value sits inside [4.79%, 6.21%], so this data does not give clear evidence the new page is different — the honest read is “we cannot yet tell them apart.” But if the old page converted at 4.0%, that falls below the whole interval, which is evidence of a genuine improvement. Same interval, two different baselines, two different decisions — and notice you never needed a p-value to get here.