SkillCoach Improves LLM Agent Skill Evaluation and Training

As Large Language Model (LLM) agents increasingly rely on reusable "skills" – encapsulating standard operating procedures, domain rules, tool workflows, and validation routines – evaluating and improving their skill-use becomes critical. Current methods often rely on coarse final verifier success, which can mask underlying issues like selecting incorrect skills, skipping steps, or faulty composition, where an agent might succeed through trial and error. Researchers have developed SkillCoach, a self-evolving rubric framework designed to provide a more nuanced evaluation of agentic skill-use. SkillCoach generates skill-grounded process rubrics directly from actual agent trajectories, assessing performance across four dimensions: skill selection, skill following, skill composition, and skill-grounded reflection. By separating process quality from task outcome, SkillCoach exposes failures hidden by final accuracy. These evolved rubrics also serve as a powerful source of process supervision, enabling the selection of high-quality training trajectories. Experiments demonstrate that SkillCoach significantly improves evaluation quality and provides stronger signals for enhancing agentic skill-use compared to outcome-only filtering.