Transformers Perform Bayesian Model Selection in Controlled Environments

A new research paper explores the capability of transformers to perform Bayesian model selection, a more advanced form of reasoning than simple Bayesian filtering. The team developed "model-selection Bayesian wind tunnels," which are controlled environments where the true posterior probabilities over different hypothesis classes are known. Within these environments, a relatively small 2.8-million-parameter transformer demonstrated remarkable accuracy, achieving near-optimal agreement with the Bayesian ideal in identifying the correct hypothesis class, even when using abstract symbols whose meanings changed per episode. This success extended to non-nested comparisons, indicating genuine model selection beyond simple biases. However, the study also uncovered a critical "perceptual access condition." When the task required arithmetic operations, such as modular addition or multiplication, the transformer's ability to perform model selection completely failed if the input symbols were opaque. This failure persisted even with significant model scaling. Conversely, the task succeeded when integer tokens were used or when opaque tokens had fixed, stable semantics. This suggests that stable semantic interpretation, rather than just integer identity, is crucial for the model to compile the necessary circuits for arithmetic reasoning and, consequently, for successful Bayesian model selection in such contexts.