From Correlation to Causation: The Two Languages

Prediction is not enough

A model trained to predict can be superb and still useless for decisions. Ice-cream sales predict drownings with high accuracy, yet banning ice cream saves no one — summer heat drives both. This is a spurious correlation: a stable statistical signal that vanishes, or even reverses, the moment you act on it. Most of machine learning answers the question *"what is the likely value of Y given that I observe X?"* Causal inference asks a fundamentally different question: *"what would Y be if I set X to a new value?"*

Pearl's ladder of causation

Judea Pearl frames the field as a three-rung ladder of causation. Each rung answers a strictly harder class of question, and no amount of data at a lower rung can, on its own, answer a higher one.

Association — seeing. P(Y | X): how does observing X change my belief about Y? This is the rung classical statistics and supervised learning live on.
Intervention — doing. P(Y | do(X)): what happens to Y if I force X to a value, overriding its usual causes? This needs the do-operator.
Counterfactual — imagining. What would Y have been for this specific unit had X been different, given what actually happened? This is the realm of counterfactual inference.

\begin{aligned} \text{Association (seeing)}:\ & P(Y \mid X) \\ \text{Intervention (doing)}:\ & P\bigl(Y \mid \operatorname{do}(X)\bigr) \\ \text{Counterfactual (imagining)}:\ & P\bigl(Y_x \mid X = x',\, Y = y'\bigr) \end{aligned}

Pearl's three rungs as formal queries: seeing P(Y|X), doing P(Y|do(X)), and imagining the counterfactual.

The jump from rung one to rung two is the heart of the field. Observational data only directly gives you the bottom rung; climbing higher requires assumptions — encoded as a graph or as an experimental design — that the data alone cannot supply.

Language one: structural causal models

A structural causal model (SCM) describes the world as a set of structural equations: each variable is written as a function of its direct causes plus an independent noise term. The equations induce a directed acyclic graph (DAG) where an arrow X → Y means "X is a direct cause of Y." The crucial move is that an SCM tells you not only the observational distribution but also what happens under surgery: intervening with do(X = x) deletes the equation for X and pins it to x, leaving everything downstream to recompute.

\begin{aligned} X &:= f_X(U_X) \\ Y &:= f_Y(X, U_Y) \end{aligned} \qquad \operatorname{do}(X = x):\ \ X := x

An SCM as structural assignments; the do-operator surgically replaces a variable's equation with a constant.

# An SCM as code: structural equations + noise
U_z, U_x, U_y = sample_noise()
Z = f_z(U_z)              # a common cause (confounder)
X = f_x(Z, U_x)           # treatment depends on Z
Y = f_y(X, Z, U_y)        # outcome depends on X and Z

# Intervention do(X = 1): replace X's equation, keep the rest
X = 1
Y = f_y(X, Z, U_y)       # Z is NOT recomputed from X

Intervention as surgery on the generative program: do(X=1) overwrites X's line; the confounder Z keeps its own equation.

Language two: potential outcomes

The potential outcomes framework (Rubin causal model) speaks of counterfactuals directly. For each unit i and a binary treatment, define two potential outcomes: Y_i(1), the outcome if treated, and Y_i(0), the outcome if untreated. The individual causal effect is Y_i(1) − Y_i(0). The fundamental problem of causal inference is that we only ever observe one of them — the other is forever missing. Causal inference is, at heart, a missing-data problem.

Because we can never see both, we retreat from the individual to a population summary: the average treatment effect (ATE), E[Y(1) − Y(0)]. The two languages are equivalent — Pearl's surgery and Rubin's potential outcomes describe the same mathematics — but they shine in different settings. SCMs make the qualitative structure of confounding visible at a glance; potential outcomes give a clean accounting frame for estimation and standard errors.

\tau_{\mathrm{ATE}} = \mathbb{E}\bigl[\, Y(1) - Y(0) \,\bigr]

The average treatment effect: the expected difference between a unit's two potential outcomes.

What this track will build

With both languages in hand, the rest of the track follows the natural workflow of an applied causal analysis. Guide 2 asks when a causal question is even answerable from data — the identification problem. Guide 3 turns identified quantities into estimates using modern machine learning. Guide 4 surveys quasi-experimental designs that manufacture identification when no clean experiment exists. Guide 5 reaches the research frontier, where ML and causality fuse: discovering graphs from data and learning representations that stay valid under intervention.