JOVANA
Explore Library Glossary Getting Started Three Levels Fields How it works Mission
Join the mission
All guides

Causal ML at the Research Frontier

Where causality meets representation learning: discovering graphs from data, learning latent causal factors, training for invariance, and reasoning counterfactually with deep models.

Learning the graph: causal discovery

Everything so far assumed someone handed you the DAG. Causal discovery algorithms try to learn that structure from data. Two broad families dominate. Constraint-based methods (PC, FCI) read conditional-independence patterns off the data and rule out any graph inconsistent with them. Score-based methods (GES, and differentiable approaches like NOTEARS) search the space of graphs to optimize a goodness-of-fit score penalized for complexity, recasting the discrete search as continuous optimization amenable to gradient descent.

X \perp\!\!\!\perp Y \mid Z \;\Longleftrightarrow\; P(X,Y\mid Z)=P(X\mid Z)\,P(Y\mid Z)

Constraint-based discovery hunts for conditional independencies; graphs sharing the same set form one Markov equivalence class that observational data alone cannot tell apart.

Causal representation learning

Classical causal inference assumes the causal variables — treatment, outcome, confounders — are given as clean columns in a table. But raw perception gives you pixels, not variables. Causal representation learning aims to recover, from high-dimensional observations, a small set of latent variables that behave like genuine causal factors: ones you could intervene on independently. It is the marriage of representation learning with structural causal models — learning what the variables even are before learning how they cause one another.

Causal representation learning maps raw pixels through an encoder into a few latent causal factors, then decodes them back — the same encoder-latent-decoder backbone as a VAE.

Diagram of a variational autoencoder: an encoder compresses the input to a latent code, a decoder reconstructs it.

This is deep water. Pure unsupervised disentanglement is provably impossible without inductive biases or extra signals, so the field leans on interventions, multiple environments, or temporal structure to make latent causal factors identifiable. The payoff would be enormous: representations that support intervention and transfer, rather than entangling everything into one opaque vector.

Invariance as a route to robustness

Why should an ML engineer who never runs an experiment care about causality? Because causal relationships are stable across environments while spurious ones are not. A predictor that leans on the cow-on-grass background fails the moment a cow stands on a beach — a textbook case of shortcut learning driving distribution shift failures. Invariant risk minimization (IRM) operationalizes this: instead of minimizing average error, seek a representation whose optimal predictor is the same across all training environments. The intuition is that an invariant predictor must be relying on genuine causes, not environment-specific artifacts.

\min_{\Phi}\;\sum_{e\in\mathcal{E}} R^{e}(\Phi)\;+\;\lambda\,\big\lVert \nabla_{w\mid w=1.0}\, R^{e}(w\cdot\Phi)\big\rVert^{2}

Invariant Risk Minimization adds a gradient-norm penalty forcing one classifier to be simultaneously optimal across every training environment, keeping only the stable causal signal.

Counterfactuals with deep models

The top rung of the ladder — counterfactual inference — asks unit-level what-ifs: *would this patient have recovered had they taken the drug they did not take?* Answering needs the full machinery of a structural causal model, executed in three steps.

  1. Abduction — use the observed facts about this unit to infer the posterior over its latent noise terms (the unit's idiosyncratic '背景').
  2. Action — apply the hypothetical intervention by surgery on the SCM's equations (e.g. do(treatment = 1)).
  3. Prediction — recompute the outcome under the modified model, holding the inferred noise fixed, to get the counterfactual outcome for this unit.
P\!\left(Y_{x}=y \mid e\right)=\sum_{u} P\!\left(Y_{x}(u)=y\right)\,P(u\mid e)

The structural counterfactual makes the three steps concrete: abduct the noise posterior from the evidence, act by fixing the treatment to x, then predict the recomputed outcome.

Deep generative models can play the role of the structural equations, letting this recipe run on images and other rich data. The same logic underwrites counterfactual approaches to offline reinforcement learning — reasoning about what reward a different action would have earned, from a fixed log of past behaviour, is exactly a counterfactual query.

Where the field is heading

The frontier is the convergence of two traditions that grew up apart: the statistical/graphical lineage of identification and estimation, and the deep-learning lineage of flexible representations. Causal ML asks them to merge — to learn structure and representation while honouring the discipline of interventions and counterfactuals. The honest state of the art is that the assumptions remain the hardest part: graphs are rarely known, instruments are scarce, invariance is fragile. The durable skill this track teaches is not any single estimator but the habit of asking what would have to be true for this number to mean what I want it to mean — and being willing to say when the data cannot answer.