LLMs Improve Evidence Use, Not Information Seeking, Under Uncertainty

This research investigates how "inference-time thinking" affects large language models (LLMs) when faced with uncertainty. By observing action preferences, thinking length, and reported confidence in a controlled two-armed bandit task, the study aimed to differentiate between improved evidence utilization and active information-seeking. Ten open-weight models were tested in both thinking and non-thinking modes. The findings indicate that thinking primarily enhances the models' ability to act on current evidence more effectively and reduces random choice noise. However, it did not lead to increased UCB-like exploration (preference for less-known options) or stronger Thompson-like exploration (choice variability increasing with uncertainty). The study also noted that thinking length increased with information-imbalanced histories, and reported confidence became more sensitive to decision difficulty, suggesting metacognitive control and monitoring processes at play, though not definitively establishing them as such.