Bayesian Curriculum Learning Optimizes LLM Reasoning by Mapping Task Manifolds

Reinforcement learning (RL) is a crucial technique for enhancing the reasoning capabilities of large language models (LLMs), with training efficiency heavily dependent on how problems are selected for optimization. Current adaptive curriculum learning methods often treat problem selection as a standard bandit problem, focusing on intermediate difficulty and overlooking the inherent structure and heterogeneity of the task space. This paper re-frames problem sampling as a "manifold-structured bandit problem" where tasks are interconnected through the LLM's latent representation space, and sampling decisions can influence how learning signals evolve across this space. To operationalize this perspective, the researchers introduce Bayesian Manifold Curriculum (BMC). BMC is a structure-aware framework that organizes problems into a hierarchical task tree and employs Bayesian learning to guide the sampling process. Empirical findings reveal that different sampling strategies create complex trade-offs between learning signal productivity, task manifold coverage (diversity), and evaluation relevance (utility). This highlights that merely prioritizing problem difficulty is inadequate for achieving robust downstream performance, underscoring the importance of incorporating structural and type-aware considerations into problem sampling for LLMs.