LaGO Improves Online Reinforcement Learning with LLM Guidance

Large Language Models (LLMs) have demonstrated significant potential in planning and sequential decision-making tasks. However, directly using LLMs as controllers in Reinforcement Learning (RL) often proves unreliable due to the need for precise action generation. This approach can lead to instability and suboptimal performance in practical applications. To address these limitations, researchers propose Latent Action Guidance for Online Reinforcement Learning (LaGO). Instead of explicit planning or control, LaGO leverages a pretrained LLM to provide a "latent action prior," which softly guides the online policy optimization process. This subtle guidance helps the RL agent learn more effectively without the rigidity of direct LLM control. Experiments conducted on both discrete-control (CLEVR-Robot) and continuous-control (Meta-World) benchmarks show that LaGO consistently enhances both the reward obtained and the success rate compared to standard PPO. Notably, LaGO significantly increased success rates, suggesting that incorporating LLM knowledge as a guiding prior can substantially improve planning and decision-making in online RL environments.