LEGO-RL Improves Reinforcement Learning for Coding Agents.

Reinforcement learning (RL) for coding agents, which often involves large language models (LLMs), typically relies on complex agent harnesses to manage tools, repository contexts, and execution feedback. However, the native environments of these harnesses are often misaligned with policy-gradient training, leading to problems like corrupted outcome signals from crashes and reward hacking, as well as discrepancies between training and inference behavior. To overcome these challenges, researchers introduce LEGO-RL, a framework designed to bridge native coding-agent harnesses with scalable policy-gradient optimization without altering their internal control flow. LEGO-RL operates on three core principles: faithful optimization through in-process LLM proxying for accurate token-level alignment; reliable execution via scalable sandbox orchestration with image caching and stage-wise defenses against reward hacking; and observable training with integrated validation, monitoring, and a live UI for granular diagnostics. Evaluations show that LEGO-RL significantly boosts the performance of the Qwen3.5-35B-A3B model across multiple coding agent harnesses, including OpenHands SDK, Claude Code, and OpenCode, on the SWE-bench Verified benchmark, while maintaining high correlation between rollout and training probabilities.