Correlation is not causation

The classic spurious correlations

In the United States, the number of people who drown each month tracks the number of ice-cream cones sold almost perfectly. Plot one against the other and you get a beautiful upward line: more ice cream, more drownings. Should we ban ice cream to save lives? Of course not. Both numbers climb in summer — when it is hot, people buy more ice cream and, quite separately, more people go swimming. The ice cream never pushed anyone into the water.

When two measurements rise and fall together, we say they are correlated, and the correlation coefficient puts a number on how tightly. But a correlation only describes a pattern in data you already have; it says nothing about what would happen if you reached in and changed one of the things. When two variables move together for a reason that has nothing to do with one causing the other, statisticians call it a spurious correlation.

Spurious correlations are everywhere and often hilarious: the number of films Nicolas Cage appears in correlates with the number of people who drown in swimming pools; per-capita cheese consumption correlates with deaths from getting tangled in bedsheets. Those are coincidences dredged out of thousands of time series. But most misleading correlations are not coincidences at all — they come from a hidden third factor, like summer heat, that drives both numbers at once.

It helps to know the handful of ways a correlation can appear with no direct cause behind it. Run through this list whenever a chart tempts you toward a conclusion.

1. Pure chance: with enough variables, some will line up by luck (the Nicolas Cage effect).
2. A common cause: a hidden third factor drives both — summer heat drives ice cream and swimming alike.
3. Reverse causation: maybe Y causes X, not the other way round (sick people rest in bed, so bed-rest can look like it causes illness).
4. Selection: the way the data was gathered forces a link that is not there in the world.

The causal question: what if we had intervened?

So how do we ever move from “these two things go together” to “this one causes that one”? The leap is from seeing to doing. Correlation lives in the world of seeing: you passively observe what already happened. Causation lives in the world of doing: you imagine reaching in and changing something on purpose, then ask what changes as a result.

Make it concrete. Your team redesigns the checkout button and notices that users who clicked the new button bought more. Tempting headline: the new button lifts sales. But you only watched what users did. Perhaps the kind of person who clicks a shiny new button was already keen to buy. The causal question is different and sharper: if we forced every user to see the new button, and — in a parallel universe — forced the very same users to see the old one, how would total purchases differ?

Statisticians mark this difference with a special piece of notation. Pr(Y | X) — read “the probability of Y given that we observed X” — is what a correlation captures. Pr(Y | do(X)) — “the probability of Y if we intervene to set X” — is what a cause is about. They are usually not equal.

Seeing X versus doing X. The left side is observation (correlation); the right side imagines an intervention (causation). Confusing the two is the original sin of data analysis.

Counterfactuals and potential outcomes

The cleanest way to define a cause is to ask a question about a world that did not happen — a counterfactual. A woman takes a headache pill and an hour later her headache is gone. Did the pill cause the relief? The honest causal question is: in the counterfactual world where she took nothing, would her headache have gone away anyway? If yes, the pill did nothing. If the headache would have lingered, the pill helped.

This idea has a precise form called the potential-outcomes framework. For every unit — a person, a user, a city — imagine two numbers that exist before we do anything. Y(1) is the outcome that unit would have if treated; Y(0) is the outcome the same unit would have if not treated. Each unit carries both potential outcomes inside it, like two sealed envelopes.

The causal effect for that one unit is simply the difference between its two envelopes. If a patient would score 3 out of 10 on a pain scale with the drug, so Y(1) = 3, and 8 out of 10 without it, so Y(0) = 8, then the drug's effect for her is 3 − 8 = −5 points of pain. We call that difference the individual treatment effect.

The individual treatment effect: one unit's outcome if treated minus the same unit's outcome if not. Causality, at its root, is a comparison of one thing with its own might-have-been.

The fundamental problem of causal inference

Here is the catch that makes causal inference a field of its own. For any single unit you can only ever see one of its two potential outcomes. Give the patient the drug and you observe Y(1) = 3; the world where she took nothing, Y(0), never happens, so you never see it. Withhold the drug and you observe Y(0); now Y(1) is the one that vanishes. The other envelope is always missing. The statistician Paul Holland named this the fundamental problem of causal inference.

Put differently, causal inference is a missing-data problem in disguise. We would love to fill in a table with two columns, Y(1) and Y(0), for every unit — but exactly half of every row is blank, and no amount of staring at the observed half will reveal the hidden one. The individual effect Y(1) − Y(0) can never be measured directly for a real person.

What we actually record is a single observed outcome that switches between the two envelopes depending on whether the unit got the treatment. Write Tᵢ = 1 if unit i was treated and Tᵢ = 0 if not:

We see Y(1) only for the treated and Y(0) only for the untreated — never both for anyone. The missing half is the counterfactual.

The average treatment effect

If individual effects are hopeless, we lower our ambition in a clever way: instead of asking what the drug did for one person, we ask what it does on average across a whole group. That quantity is the average treatment effect, or ATE — the average of Y(1) − Y(0) over everyone. It is the workhorse of practical causal inference.

Writing E for “average over the whole population”, the ATE is the average of the treated-world outcome minus the average of the untreated-world outcome:

We can never average a column we half-observe — but we can sometimes estimate each piece, E[Y(1)] and E[Y(0)], from different groups of units.

Here is the magic of a randomized experiment. Flip a coin for each unit: heads go to the treatment group, tails to the control group. Because the coin ignores everything about each person, the two groups are, on average, alike in every way — age, motivation, the weather, things you never even measured. So the treated group's average outcome is a fair stand-in for E[Y(1)], and the control group's average stands in for E[Y(0)]. Their difference estimates the ATE.

In a randomized experiment, the simple difference in group averages is an unbiased estimate of the average treatment effect. This one line is why randomized trials are the gold standard.

Concretely: split 2,000 users at random, show 1,000 the new checkout button and 1,000 the old one. If 12.0% of the new-button group buys and 10.0% of the old-button group buys, your estimated ATE is 12.0% − 10.0% = 2.0 percentage points. Because the split was random, you can defend those 2 points as a causal effect, not merely a correlation.

import pandas as pd

# one row per user: group is 'treat' or 'control', bought is 0 or 1
rates = df.groupby("group")["bought"].mean()
ate = rates["treat"] - rates["control"]
print(ate)   # 0.12 - 0.10 = 0.02  ->  a 2-point lift

Why naive group comparisons mislead

Now strip away the randomization — the situation you face with almost all data that simply arrived: server logs, hospital records, a survey. People chose, or were chosen, into the groups. When the treated and untreated differ before any treatment, the plain difference in their averages is no longer the ATE. It is the ATE plus a contaminating term.

The number you can compute equals the number you want only when the bias term is zero. Randomization forces it to zero; raw observational data almost never does.

The most common source of that bias is a confounder: a variable that influences both who gets the treatment and what the outcome is. Summer heat is the confounder behind ice cream and drowning — it raises both. In the checkout example, “how keen the user already was to buy” may drive both clicking the new button and purchasing. A confounder forges a correlation with no causal arrow between the two things you care about.

A close cousin is selection bias: the very act of choosing who lands in each group bakes in a difference. If hospitals send their sickest patients to a new treatment, the treated group looks worse afterward — not because the treatment hurts, but because they started sicker. Compare the groups naively and you might “prove” that a life-saving therapy is deadly.

The most dramatic warning is Simpson's paradox, where a comparison can reverse direction once you split by a confounder. A famous kidney-stone study found that Treatment A had a lower overall success rate than Treatment B — yet A won for small stones and A also won for large stones. How can A be better in every subgroup yet worse overall? Because doctors gave the tougher Treatment A to the hard cases (large stones), so A's overall number was dragged down by the patients it was handed, not by the treatment itself.

The road ahead: how we still make causal claims

If correlation is so treacherous, can we ever earn the word “cause”? Yes — by one of two routes. The first is to manufacture the missing counterfactual ourselves through randomization, the gold standard you met above and the subject of the experiment-design track. When you can flip the coin, the bias term is gone by construction, and a difference in means simply is a causal effect.

Often you cannot randomize — it would be unethical, illegal, impossible, or far too slow. You cannot randomly assign people to smoke for thirty years. The second route is to reason carefully about how the data came to be, drag the assumptions out into the open, and use designs that mimic an experiment from purely observational data. That is the rest of this track.

1. Draw the causal diagram. A causal DAG of arrows lets you see which variables to control for — and, surprisingly, which to leave alone (the next guide).
2. Borrow nature's coin flips. Natural experiments — instrumental variables, difference-in-differences, and regression discontinuity — exploit quirks of the world that assign treatment almost at random.
3. Build a fair comparison after the fact. Matching and propensity scores pair treated units with similar untreated ones to balance confounders.
4. Then stress-test the answer: how fragile is it to a confounder you forgot to measure?

Before you move on, here is a portable checklist. Run it on the next “X causes Y” headline you meet — in a paper, a press release, or your own dashboard.

1. Was anyone randomized? If yes, the claim stands on solid ground. If no, stay skeptical.
2. What is the obvious common cause? Name a confounder that could create the link all by itself.
3. Could the arrow run backwards? Maybe the outcome causes the supposed “cause”.
4. How were the groups selected? Look for selection bias in who ended up being measured.
5. Does the effect size matter? A real but tiny effect rarely justifies a big decision.

Correlation is not causation — but with the right question (a counterfactual), the right target (the average treatment effect), and an honest account of confounding, causation is something we can still pursue. The guides ahead hand you the tools. Keep the spurious ice cream in mind: the goal is never to fear correlations, but to ask, every single time, what is really making these numbers move together.