APEx Distills Agent Experience for Adaptive Research QA

Deep research agents, which augment large language models with external tools, are designed to answer complex, long-horizon questions through multi-turn reasoning. For these agents to continually improve, learning from past experiences is crucial. However, existing methods either retrieve overly verbose task-specific traces, which can burden decision-making, or distill procedural skills that remain disconnected from how the agent's policy adapts.This research introduces APEx (Distillation of Agent Procedural Experience), a hierarchical framework that addresses these limitations. APEx structures an agent's interaction history into two levels: instance-level trajectory memories and category-level procedural skills. These are then tightly coupled through a closed-loop architecture comprising three modules: an Executor, a Distiller, and a Planner.The three modules are optimized using a three-stage alternating GRPO (Generalized Policy Optimization) training paradigm. This allows for reward-guided skill distillation, moving beyond fixed-prompt generation. During testing, the distilled skills act as procedural priors, guiding the Planner's online adaptation through skill-guided test-time reinforcement learning. This enables ground-truth-free self-improvement, with skill-alignment regularization preventing policy drift. APEx has achieved state-of-the-art performance across seven benchmarks, outperforming GPT-5.4 by 14.7 points and the strongest memory-augmented baseline by 3.0 points.