Randomness, variables & distributions

What “random” really means (and what it doesn’t)

In everyday speech, “random” means weird, unfair, or out of nowhere — “what a random thing to happen.” In statistics it means something much more precise and much more useful: an outcome you cannot predict on any single occasion, but that follows a stable, knowable pattern when you watch it many times. The single flip of a coin is unpredictable. Ten thousand flips of the same coin are extraordinarily predictable: just about half will land heads.

That is the single most important reframe in this guide. Randomness is not the absence of pattern — it is a pattern that only shows up in the aggregate. A casino cannot tell you whether the next spin of the roulette wheel will win or lose; yet it can tell you, almost to the dollar, how much it will earn from a million spins. Data science lives in exactly this gap: individual events are uncertain, but their collective behaviour is law-like enough to bet a business on.

Randomness can come from two quite different places, and it helps to keep them apart. Some of it is genuine chance built into the process — radioactive decay, quantum noise, the shuffle of a well-mixed deck. But most randomness in data is really our own ignorance: which customer clicks next is, in principle, determined by their mood, their screen, the weather, a hundred causes we never measured. Because we cannot see those causes, the outcome behaves, from where we sit, exactly like chance. For doing statistics, the source rarely matters — we model both the same way.

Probability in one minute: long-run frequency

A probability is just a number between 0 and 1 that measures how often something happens in the long run. Zero means it never happens; one means it always happens; one-half means it happens about as often as not. That is the whole idea. When we say a fair six-sided die has probability 1/6 of landing on a 4, we are making a concrete, checkable claim: if you roll it 6,000 times, you should see roughly 1,000 fours.

This is the frequentist reading of probability (named because it ties probability to long-run frequency): the probability of an event is the fraction of the time it would occur if you could repeat the situation over and over under the same conditions. For a setup where every outcome is equally likely — a fair die, a shuffled deck — you can compute it just by counting.

For equally likely outcomes, probability is a counting exercise. Rolling a 4: one favourable face out of six, so 1/6 ≈ 0.167.

Two rules will carry you a long way. First, every probability sits between 0 and 1 — you will never see −0.3 or 1.4. Second, if you list every possible outcome of one trial so they do not overlap and nothing is left out, their probabilities add up to exactly 1. The die’s six faces each have probability 1/6, and 6 × 1/6 = 1: something must happen. The set of all possible outcomes is called the sample space, and a “something we care about” — like “the roll is even” — is an event, simply a collection of outcomes.

Two honest cautions. The long-run picture is an idealisation: you never actually roll a die infinitely many times, and a fair coin can perfectly well give you seven heads in a row — that is randomness, not a broken coin. And probability says nothing about which single outcome you will get next. Anyone who tells you a coin is “due” for tails after a streak of heads has fallen for the gambler’s fallacy; the coin has no memory.

If you want the gentle build-up of the rules — sample spaces, events, and how probabilities combine — see probability basics and sample spaces and events. For our purposes, the one-minute version above is enough to get moving.

Conditional probability & independence, with a concrete table

Most real questions are not “how likely is this?” but “how likely is this, given what I already know?” That second kind is a conditional probability: the chance of one thing once you have learned that another thing is true. The notation P(A | B) is read “the probability of A given B,” and the vertical bar means “given.” It is the engine behind almost every prediction you will ever make from data.

Let’s make it real with a small marketing dataset. Imagine 1,000 newsletter subscribers. We sent them an email and later recorded who opened it and who went on to buy. The four boxes below — a contingency table, which just cross-tabulates two yes/no variables — hold the whole story.

                 bought   did not buy   row total
opened email        80          320          400
did not open        20          580          600
--------------------------------------------------
column total       100          900        1,000

Start with the plain, unconditional buy rate. Out of 1,000 people, 100 bought, so P(buy) = 100/1,000 = 0.10, a 10% chance. Now condition on opening the email. Among the 400 openers, 80 bought, so P(buy | opened) = 80/400 = 0.20 — a 20% chance. Among the 600 non-openers, only 20 bought, so P(buy | did not open) = 20/600 ≈ 0.033, about 3.3%. Knowing one extra fact — did they open it? — moved our estimate of the buy probability from 10% to either 20% or 3.3%. That movement is the whole value of conditioning.

The recipe: the chance of both, divided by the chance of the thing you conditioned on. Here P(buy and opened)=80/1000 and P(opened)=400/1000, so the 1000s cancel and you get 80/400.

Now the flip side. Two events are independent when learning one tells you nothing about the other — the condition simply does not move the probability. Formally, A and B are independent if P(A | B) = P(A), which is equivalent to saying the joint probability factorises:

Independence as multiplication. Two fair coin flips: P(heads then heads) = 0.5 × 0.5 = 0.25, because the first flip tells you nothing about the second.

Our email and buying are clearly not independent: if they were, P(buy | opened) would equal the plain P(buy) of 10%, but it is 20%. Opening and buying travel together. Two fair coin flips, by contrast, are independent — the coin has no memory, so the second flip ignores the first. Drawing two cards from a deck without putting the first back is not independent: removing the ace of spades changes what is left.

See conditional probability and independence for more worked cases. Conditioning the right way — and noticing when independence quietly fails — is, honestly, a large part of what separates careful analysts from confident ones.

Random variables: turning outcomes into numbers

So far our outcomes have been words — heads, opened, a 4. To do arithmetic with chance, we need numbers. A random variable is simply a rule that attaches a number to each outcome of a random process. The classic everyday example: flip three coins and let X be the number of heads. Whatever happens, X is one of 0, 1, 2, or 3, and which one you get is, before the flips, a matter of chance. We usually write the variable as a capital letter (X) and a particular value it takes as a small one (x).

List every value the variable can take alongside the probability of each, and you have its probability distribution — the complete fingerprint of a random variable. For three fair coins there are eight equally likely outcomes, and counting how many give each number of heads yields:

X = number of heads in 3 coin flips

 x   outcomes that give it      P(X = x)
 0   TTT                        1/8 = 0.125
 1   HTT, THT, TTH              3/8 = 0.375
 2   HHT, HTH, THH              3/8 = 0.375
 3   HHH                        1/8 = 0.125
                          total: 8/8 = 1

Random variables come in two flavours. A discrete one can take only separated values you could in principle list — number of heads, number of support tickets today, a star rating from 1 to 5. A continuous one can take any value in a range — a person’s exact height, the precise time until your next website visit, a temperature. For a discrete variable we can give a probability to each value; for a continuous one any exact value has probability essentially zero (no one is exactly 170.000… cm), so we instead talk about the probability of landing in an interval, and we picture it as area under a curve. We will see that curve in a moment with the bell shape.

Why bother turning outcomes into numbers? Because numbers let us summarise. Instead of carrying around the whole eight-row table, we can ask two simple questions of any random variable: where is it centred, and how spread out is it? Those two numbers — coming up next — describe a huge amount of what we ever need to know.

More on this idea lives at random variable and probability distribution (and a more mathematical treatment at random variable in the probability glossary).

Center & spread: mean, variance, standard deviation

The center of a random variable is its expected value, also called its mean — the long-run average value you would get if you sampled it over and over. It is not the most likely single value and it need not even be a value the variable can take; it is the balance point of the distribution. For a fair die, average the six faces: (1+2+3+4+5+6)/6 = 3.5. You will never roll a 3.5, yet 3.5 is exactly where thousands of rolls will average out.

When the outcomes are not equally likely, you weight each value by its probability before adding — a probability-weighted average. For our three-coin variable X, that is 0×(1/8) + 1×(3/8) + 2×(3/8) + 3×(1/8) = 12/8 = 1.5 heads on average, which makes perfect sense: three coins, each contributing half a head on average.

Expected value: run through every possible value, multiply by its probability, and add. For equally likely outcomes this is just the ordinary average.

But the center alone can fool you. Picture two investment funds, each with an average annual return of 5%. Fund A returns a steady +5% every year. Fund B returns +25% in good years and −15% in bad years, fifty-fifty. Their means are identical (0.5×25 + 0.5×(−15) = 5), yet they are wildly different things to own. We need a second number for how much the outcomes bounce around the center. That number is the variance, and its more readable cousin the standard deviation.

Variance is the average squared distance from the mean. Why squared? Distances above and below the mean would otherwise cancel to zero, and squaring also punishes big misses more than small ones. For Fund A every outcome is exactly the mean, so the variance is 0 — no surprises. For Fund B each outcome is 20 percentage points from the mean, so the variance is 0.5×(20²) + 0.5×(20²) = 400.

Variance is the expected squared distance from the mean μ (Greek “mu”). The standard deviation σ (“sigma”) is its square root, which returns to the original units.

Squaring inflates the units — variance 400 is in “percentage-points-squared,” which means nothing to a human. So we take the square root to get back to plain percentage points: the standard deviation of Fund B is √400 = 20%, while Fund A’s is √0 = 0%. That is the headline number a person can feel: Fund B’s yearly return typically lands about 20 points away from its 5% average. Standard deviation is the everyday measure of spread, risk, and noise — keep it; you will use it constantly.

Deeper treatments: expected value and expectation, plus variance and standard deviation (and a more formal standard deviation entry).

A small zoo of distributions

A handful of named distributions describe a remarkable share of the random things you will ever model. Think of them as templates: recognise the kind of question you are asking, and the right shape comes with formulas, intuition, and ready-made tools attached. Here are the five worth knowing on day one.

1. Bernoulli — one yes/no trial with success probability p. A single coin flip, one visitor who either clicks or doesn’t. The atom from which the others are built.
2. Binomial — the number of successes in n independent Bernoulli trials. Send 10 emails that each open with probability 0.2; how many of the 10 open? Our three-coin “number of heads” was a binomial with n = 3, p = 0.5.
3. Poisson — the count of rare events in a fixed window of time or space, when each can happen independently. Support tickets per hour, typos per page, customers arriving per minute.
4. Uniform — every outcome in a range is equally likely. A fair die, a well-built random-number generator picking between 0 and 1. Flat and featureless on purpose.
5. Normal — the continuous bell curve: symmetric, single-peaked, with most of the mass near the mean and thin tails far away. Heights, measurement errors, and — crucially — averages of many things.

Choosing a distribution is really about matching the story of how your data is generated, not about hunting for the curve that fits a histogram best. Ask: Is each observation a single yes/no? Then Bernoulli. Counting successes among a fixed number of tries? Binomial. Counting rare events over an interval? Poisson. Is everything equally likely? Uniform. And whenever you are looking at an average or a sum of many small influences, suspect the normal — which is exactly the next section.

Each animal in the zoo has its own page: Bernoulli, binomial, Poisson, uniform, and normal.

Why the normal distribution shows up everywhere

Of all the shapes, the normal — the bell curve — is the one you will meet most often, and there is a deep reason why. It is not that nature has a fondness for bells. It is that whenever a quantity is the sum or the average of many small, independent influences, that total tends to look normal no matter what the individual pieces look like. A person’s height is many genes plus nutrition plus sleep plus chance, added up — and heights are bell-shaped. A measurement error is the sum of dozens of tiny jitters — and errors are bell-shaped. This near-magical fact has a name: the Central Limit Theorem.

Here is the demonstration in miniature. A single die roll is uniform — flat, not bell-shaped at all. But roll ten dice and take their average, then do that thousands of times and plot all those averages: the picture is a clean bell, centred on 3.5. The flatness of the individual roll has vanished; averaging has manufactured a normal shape out of a non-normal ingredient. Try it below.

The bell is convenient because once you know its center (the mean μ) and its spread (the standard deviation σ), you know it completely. That gives the famous empirical rule, true for any normal distribution: about 68% of values fall within one standard deviation of the mean, about 95% within two, and about 99.7% within three. If adult heights average 170 cm with a standard deviation of 7 cm, then roughly 95% of people fall between 156 and 184 cm — and someone at 200 cm is more than four standard deviations out, genuinely rare.

Standardizing: subtract the mean and divide by the standard deviation to express any value as “how many standard deviations from average.” The 200 cm person has z = (200−170)/7 ≈ 4.3.

That last point — that averages become normal and predictable as samples grow — is the hinge the whole field turns on. It is what lets a survey of a few thousand people speak for millions, and what gives margins of error their width. We have only previewed it here; the next guide makes it the main act. Read on at the normal distribution (and the normal distribution for the math), the central limit theorem (with its formal statement), and its quieter sibling the law of large numbers.