New Framework Boosts AI Agent Skill Learning and Utilization

In agentic reinforcement learning, leveraging skill memories can enhance performance by providing textual guidance from past experiences. However, these retrieved skills are not always perfectly applicable; a skill beneficial in one situation might be misleading in another. This challenge undermines the common assumption that a skill-conditioned prompt can serve as a fixed, reliable teacher. To address this, the UCOB framework (Utilize and Evolve Agentic Skills via Credit-Aware On-Policy Bidirectional Self-Distillation) has been developed. UCOB treats both skill-conditioned and no-skill prompts as different contextual views of the same model. It compares their "return-to-go" within the same task and anchor state, then uses the view yielding a higher return as the local teacher. This credit-aware local signal helps the agent internalize effective skill-conditioned behaviors, rectify instances of misleading skill application, and guide updates to skill memory, utility-aware retrieval, and reflection-based self-training. Experiments across various agentic tasks, including ALFWorld, WebShop, and Search-QA, demonstrate that UCOB significantly outperforms skill-free RL, other skill-memory baselines, and self-distillation methods, achieving substantial performance gains.