New Sampling Methods Improve Causal Bandits with Non-Manipulable Variables

This paper explores contextual causal bandits in scenarios where some variables cannot be directly manipulated, yet they provide crucial information for decision-making and influence rewards. The research adopts a Bayesian framework, assuming a known causal graph without latent confounding, where the unknown parameters are the conditional probability tables of the observational distribution. This setup allows observations from one intervention to inform reward estimates for other interventions by leveraging shared causal mechanisms. The authors develop causal adaptations of two prominent bandit algorithms: Thompson Sampling and Information-Directed Sampling (IDS). For Thompson Sampling, they establish an entropy-dependent sublinear Bayesian regret bound. For IDS, they derive a similar entropy-dependent regret bound that explicitly accounts for errors introduced by Monte Carlo approximations. When exact quantities are available, this bound recovers the standard sublinear IDS rate. High-probability confidence bounds are also provided for the Monte Carlo estimates. Experimental results on several synthetic causal bandit tasks demonstrate that these proposed causal methods significantly outperform both causal and non-causal baselines. This superior performance is attributed to their more effective exploitation of information shared across different interventions, leading to accelerated identification of optimal decisions.