SCOUT Improves Sparse-Reward RL with Per-Context Curricula.

Reinforcement learning (RL) in sparse-reward environments often struggles because agents rarely encounter successful states, making learning difficult. Reset curricula, which start training from easier intermediate states (scaffolds), can help. However, most existing curricula use a single, shared schedule for removing this assistance, which can be inefficient or even fail when different task instances, or "contexts," learn at varying rates. Researchers introduce SCOUT, an innovative online reset controller that assigns a unique curriculum to each context. SCOUT operates without modifying the reward function, optimizer, or underlying learner, relying solely on binary rollout success. It intelligently removes assistance after consistent success, reinstates it upon failure, and cautiously attempts harder starting points if progress stagnates. Experiments across various navigation and manipulation tasks demonstrate that SCOUT significantly improves learning and enables success in scenarios where unassisted training fails. Crucially, in situations designed to create pacing conflicts, global schedules left some groups unsolved, whereas SCOUT successfully addressed all, highlighting its ability to adapt to individual learning needs within a diverse set of contexts.