Causal Decision-Making Taxonomy
Context
Ge2026 - A Review of Causal Decision Making organizes a broad literature by separating causal decision making into three tasks and six data paradigms. This annex keeps that routing schema compact so future notes can point to it without repeating the full survey taxonomy.
Formal Statement
Let S be state or context, A an action, R a reward or outcome, b a behavior policy, and pi a target policy.
The three tasks are:
- CSL: learn or posit causal structure over variables such as
S,A, mediators, confounders, andR. - CEL: identify and estimate causal effects such as
E[R | do(A=a)],ATE, heterogeneous effects, and mediation effects. - CPL: evaluate or optimize policies, e.g. estimate
eta(pi) = E_{s~G} V^pi(s)or learnpi_hatwith small regret relative to an optimal policy.
The six paradigms are:
| Policy/data regime | Independent states | Markovian transitions | Non-Markovian/history dependence |
|---|---|---|---|
| Fixed policy / offline | Paradigm 1: i.i.d. treatment or contextual decision data | Paradigm 2: logged MDP data | Paradigm 3: panel data, DTRs, offline history-dependent RL |
| Adaptive policy / online | Paradigm 4: bandits and contextual bandits | Paradigm 5: online RL in MDPs | Paradigm 6: POMDPs, DTR bandits, and history-dependent online learning |
The baseline identification assumptions are:
- SUTVA/consistency: the observed reward equals the potential reward for the action actually assigned, often written
R_i = sum_a R_i(a) 1{A_i=a}, and one unit’s outcome is not changed by other units’ treatments. - No unmeasured confounders:
R(a)is conditionally independent ofAgivenS. - Positivity/overlap: each relevant action has nonzero probability under the observed data for the states where the target policy needs support.
Derivation / Construction
To route a new CDM paper, first ask whether the data are fixed-policy or adaptively collected. Then ask whether each next state/reward is independent, Markovian in the current state-action pair, or history-dependent. Finally ask whether the paper primarily learns structure, estimates effects, evaluates a policy, optimizes a policy, or handles an assumption violation.
This construction also explains why familiar estimators reappear across fields. Direct, inverse-probability-weighted, and doubly robust estimators show up in single-stage CEL and then reappear in OPE. Q-learning, fitted-Q evaluation, and fitted-Q iteration are policy-learning analogues once the decision target is a value function rather than a one-step treatment effect.
Implications
The taxonomy prevents a useful mistake: treating all decision-making papers as either “causality” or “RL” without recording the data-generating assumptions. Offline RL, online bandits, causal effect estimation, panel methods, and dynamic treatment regimes can share estimators while requiring different support, confounding, and dependence assumptions.
It also identifies where assumptions fail. Unmeasured confounding calls for instruments, proxies, sensitivity analysis, or data integration. Interference calls for exposure mappings, network or cluster structure, or multi-agent modeling. Positivity failures call for trimming, matching, partial identification, smoothing, pessimism, or new data collection.