From identification to estimation
Guide 2 reduced the ATE to an adjustment formula built from conditional expectations like E[Y | X, Z] and the treatment probability. Those conditional functions — call them nuisance functions — are exactly what supervised learning is good at. The temptation is obvious: fit a strong random forest or gradient boosting model for each piece, plug them into the formula, and read off the effect. This plug-in approach is also exactly where naive causal ML goes wrong.
The propensity score
A foundational tool is the propensity score: e(Z) = P(X = 1 | Z), the probability of receiving treatment given the covariates. Rosenbaum and Rubin's insight is that conditioning on this single scalar is enough to remove confounding from a high-dimensional Z — it is a balancing score. Propensity score matching uses it directly: pair each treated unit with control units of similar propensity, so that within a matched pair the only systematic difference is treatment itself, mimicking a small randomized experiment.
The propensity score: the probability of receiving treatment given the covariates.
Why naive plug-in is biased
Flexible ML models are tuned to predict well, which means they trade a little bias for a lot of variance reduction — regularization deliberately shrinks estimates. That bias is harmless for prediction but poison for a plug-in causal estimate: the model's regularization bias leaks directly, at first order, into the effect estimate. The result is a confidence interval centred in the wrong place, no matter how large your sample. The deeper diagnosis is that the plug-in estimator is sensitive to small errors in the nuisance functions.
Curves showing bias falling and variance rising with model complexity, with total error minimized in between.
Double machine learning
Double machine learning (DML) fixes this with two ideas. First, Neyman orthogonality: construct a score for the effect whose derivative with respect to the nuisance functions is zero at the truth. Small nuisance errors then hurt the estimate only at second order — their first-order influence cancels. Concretely you residualize: predict Y from Z, predict X from Z, and regress the Y-residuals on the X-residuals (the Robinson partialling-out trick). Second, cross-fitting: estimate the nuisances on one fold and evaluate the score on a held-out fold, so the same data point never both trains the nuisance and contributes to the effect — killing the overfitting bias that cross-validation readers will recognize.
Double machine learning: regress the outcome residual on the treatment residual after partialling out the covariates with flexible ML.
# Double ML for a partially linear model Y = theta*X + g(Z) + e
for fold k in cross_fitting_folds:
train, eval = split_excluding(k), fold(k)
m_Y = fit_ml(Z -> Y, on=train) # outcome model
m_X = fit_ml(Z -> X, on=train) # propensity / treatment model
rY = Y[eval] - m_Y.predict(Z[eval]) # outcome residuals
rX = X[eval] - m_X.predict(Z[eval]) # treatment residuals
# theta_hat solves the orthogonal score: regress rY on rX
theta_hat = sum(rX * rY) / sum(rX * rX)Beyond the average: heterogeneous effects
An average can hide everything that matters. A drug that helps the young and harms the old may show an ATE near zero. The conditional average treatment effect (CATE), τ(z) = E[Y(1) − Y(0) | Z = z], restores the heterogeneity: the effect as a function of who you are. Orthogonal, cross-fitted machinery extends to CATE — causal forests, the R-learner, and DR-learner build on the same Neyman-orthogonal scores so that a flexible model can estimate τ(z) without inheriting nuisance bias.
The conditional average treatment effect: the average effect for the subgroup with covariate profile z.
CATE is also the bridge to decisions: knowing τ(z) tells you whom to treat. Guide 4 turns that into targeting policies via uplift modeling; here, just hold the principle that estimating a per-individual effect is harder than an average and demands even more discipline about overlap and orthogonality.