EvoHarness-RL Enables LLM Agents to Learn Self-Evolving Runtime Harnesses.

Long-horizon LLM agents often struggle with managing external state, tracking progress, and effectively using tools across complex interactions. Current solutions typically rely on manual engineering, prompts, or heuristics for these "harness" functions, leading to inefficiencies and limited adaptability. EvoHarness-RL introduces a novel approach where agents learn and deploy their own harness policies. It exposes Belief, Progress, and Experience (BPE) as policy-facing harness states, enabling agents to construct and update this external state dynamically. The framework uses supervised fine-tuning for basic harness actions and cost-aware GRPO for selective state management. Empirical results on ALFWorld show EvoHarness-RL achieving high success rates. It demonstrates "harness annealing," where agents internalize common patterns and shift towards selective external state access, and "harness evolution," where the harness refines into a compact, task-adaptive state. This suggests that trainable policies for external workspaces significantly benefit long-horizon agents beyond just stronger tools or larger memories.