SkillHEX Improves Agent Skills Through Autonomous Exploration and Exploitation

Large Language Model (LLM) agents rely on "skills" for reusable procedural knowledge, but manually maintaining these skills is costly and unscalable. Real-world deployments demand autonomous skill evolution, often under tight interaction budgets and without traditional training or validation sets, leading to a severe sparse reward problem where outcomes don't clearly indicate failure causes. Existing methods that greedily refine a single skill often fall into an "exploitation trap," exhausting limited trials on unproductive paths due to early misdiagnoses. SkillHEX addresses this by introducing a novel closed-loop framework. It combines hypothesis-driven self-verification with an evidence-guided tree search mechanism. SkillHEX translates falsifiable failure hypotheses into executable tests, generating diagnostic evidence that acts as a dense reward without requiring additional environment attempts. This evidence then guides a search across multiple skill-revision branches, dynamically balancing the exploitation of supported edits with the exploration of plausible alternatives. Evaluated on 87 tasks from SkillsBench, SkillHEX significantly outperforms other self-evolving methods, achieving higher pass rates with limited budgets using models like GPT-5.3-Codex and Claude Opus 4.7.