New RL Method Boosts LLM Agent Performance on Complex Tasks.

A new research paper introduces ProGPO (Progress- and Reliability-Oriented Group Policy Optimization), an innovative method for enhancing the performance of large language model (LLM) agents in complex, multi-step tasks. Traditional group-based reinforcement learning (RL) for LLMs often struggles with fine-grained policy updates due to challenges in forming consistent and informative groups for comparison. ProGPO addresses these issues by maintaining exact-prefix action comparisons, ensuring that actions are evaluated within their correct historical context. It complements sparse peer comparisons with "transition credit" derived from rollout-based state potentials, which are estimated more reliably through semantic expansion and inverse-variance fusion across different history depths. This approach allows for more stable and effective step-level learning. Evaluations on challenging agentic tasks like ALFWorld and WebShop, using Qwen2.5-1.5B-Instruct, demonstrate that ProGPO significantly improves upon existing agentic RL baselines. The method also shows scalability with larger models, indicating its potential for developing more capable and reliable AI agents.