Transformers Act as Bayesian Experimenters for ATE Estimation

Estimating Average Treatment Effects (ATE) efficiently in adaptive experiments requires careful allocation of treatments that balances valid inference with statistical efficiency. The ideal approach, a covariate-dependent Neyman rule, relies on unknown outcome variances. This paper explores a novel method where transformers are trained to act as 'Bayesian in-context experimenters,' effectively amortizing the sequential process of variance estimation and treatment allocation. These transformer policies learn to imitate a Bayesian posterior Neyman teacher, which dynamically updates nonparametric beliefs about potential outcomes based on experimental history to assign optimal treatment probabilities. This design converges to the oracle rule, significantly improving ATE inference efficiency. To address the challenge of unknown outcome smoothness, the researchers combine multiple smoothness-indexed experimenters using a mixture-of-experts transformer. The gating mechanism acts as a hierarchical posterior, concentrating on the most appropriate expert, thereby enabling adaptive and near-oracle performance.