Offline RL Controls LLM Agent Harnesses for Better Performance.

Large language model (LLM) agents are typically enhanced by modifying prompts, models, or hand-written workflows, with the surrounding execution harness often considered fixed infrastructure. This research challenges that view, proposing that the harness itself can be a learnable control layer. The authors formalize harness operation as a finite-horizon Harness MDP, where a lightweight controller makes structural execution decisions while the core LLM remains unchanged. This controller is trained using offline reinforcement learning, specifically advantage-weighted regression, relying only on terminal task-rubric rewards. A key innovation is the separation of final task quality from a "Harness Maturity Score," which evaluates whether the harness follows reliable execution patterns. This distinction allows for learning process behavior even when immediate final-quality gains are limited by the offline buffer. Experiments across various controlled domains and public benchmarks show that the learned controller consistently improves verification behavior and selectively enhances final task quality, particularly in retail, database, and coding tasks.