causal reinforcement learning

Ordinary RL learns correlations from the data it happens to collect: do this, that tends to follow. Causal reinforcement learning insists on the stronger notion of cause and effect — what would happen if I intervened, even in a way I have never tried. The distinction matters because a policy built on correlation can be fooled by confounders: a hospital agent might see that a treatment correlates with recovery, when really the healthier patients were the ones given it.

The framework borrows from causal inference: structural causal models and the language of interventions and counterfactuals let an agent reason about do-style actions rather than mere observation. This pays off in several places — handling confounded offline data where the logging policy depended on hidden factors, transferring knowledge across environments that share causal mechanisms but differ in surface statistics, and exploring more efficiently by targeting the variables that actually drive reward.

Causal RL is a young and ambitious thread that promises better transfer, sample efficiency, and trustworthiness, but it leans on causal knowledge that is itself hard to obtain — discovering the causal graph from data is famously difficult, and wrong assumptions can mislead. It connects RL to structural causal models and the broader project of causal representation learning.