SkillAudit Evolves Agent Skills Without Ground-Truth Feedback

AI agent skills, which are structured procedures guiding Large Language Models (LLMs) through specialized workflows, often require continuous refinement after deployment. Real-world usage frequently uncovers edge cases, API changes, or operational constraints that were not apparent during initial development. However, traditional skill evolution methods typically rely on privileged feedback, such as hidden test scores or environment rewards, which are often unavailable in practical deployment scenarios where only a task description and workspace data exist. This paper introduces SkillAudit, a framework designed to overcome this limitation by evolving agent skills without needing ground-truth feedback. Its core innovation is 'paired trajectory auditing,' where the same task is executed both with and without a candidate skill at each iteration. This approach isolates how the skill specifically alters the agent's behavior, providing insights into its impact. To translate these behavioral differences into actionable editing guidance, SkillAudit employs Process-Aligned Contrastive Evaluation (PACE). PACE consists of evaluators that map observed trajectory divergences to diagnostic signals, which are then linked to specific sections within the skill document. A fixed structural verifier, derived from the task specification, ensures that any proposed updates adhere to task constraints and prevents harmful modifications. SkillAudit routes edits through two pipelines: 'Refine' for broadly useful skills and 'Repair' for passages that conflict with the task, demonstrating significant performance gains across various professional domains.