ASK+ Improves SLM Guidance for RL Agents in Partial Observability

Reinforcement learning agents often struggle in environments where information is incomplete, known as partial observability. While small language models (SLMs) possess broad reasoning capabilities, integrating their guidance into such settings has been challenging, with previous "uncertainty-gated" methods showing minimal SLM contribution. This failure was traced not to SLM capacity, but to insufficient context provided in the prompts. Researchers introduce ASK+, a novel approach that supplies the SLM with rich, trajectory-aware context, including partially revealed maps, visited positions, and action history, alongside structured chain-of-thought reasoning. This transforms the SLM from a passive checker into an active, corrective consultant. ASK+ demonstrates substantial performance gains across various environments, with a 2B parameter model matching or exceeding a 4B model, confirming that prompt design and selective gating are more critical than model scale for effective guidance in partially observable RL.