EnvRL Framework Boosts LLM Agent Performance in Complex Tasks

This research introduces EnvRL, a novel framework designed to improve the performance of Large Language Models (LLMs) when operating as agents in complex, long-horizon tasks. Traditional reinforcement learning (RL) often struggles with sparse rewards in such scenarios, overlooking valuable information embedded in the agent's interactions with its environment. EnvRL addresses this by treating interaction experience as an implicit supervisory signal, allowing the agent to build a more accurate internal model of the environment's dynamics. The framework integrates two auxiliary objectives—state prediction and inverse dynamics—which are optimized alongside the primary RL objective. Experimental results on benchmarks like ALFWorld and WebShop demonstrate that EnvRL significantly boosts success rates compared to RL-only baselines. For instance, it improved a Qwen-2.5-1.5B-Instruct model's success rate from 72.8% to 77.4% on ALFWorld, showcasing its potential for more robust and effective AI agents.