ToolAnchor Boosts AI Agent Tool-Use with Counterfactual Context

Large language model agents, while adept at complex tasks, often struggle when new tools are introduced to their environment, tending to rely on previously learned toolsets. This phenomenon, termed "behavioral inertia," prevents agents from fully leveraging new capabilities. A new framework, ToolAnchor, addresses this by strategically inserting counterfactual contexts into the agent's decision-making process. ToolAnchor uses teacher models to hypothesize these alternative contexts, which are then verified through student model rollouts. Successful interventions are subsequently internalized by the agent through a targeted post-training process. This approach allows agents to break free from established patterns and effectively incorporate novel tools. Evaluations across various AI assistant and search tasks demonstrate that ToolAnchor significantly improves agent performance when faced with expanded toolsets. This work offers a scalable solution for dynamic adaptation in agentic reinforcement learning, bridging the gap between static training and the need for flexible tool integration.