New Taxonomy Pinpoints AI Agent Failure Origins for Better Repairs

A new research paper addresses the challenge of diagnosing failures in AI agent systems. Current evaluation methods often only identify system-level outcomes, making it difficult to pinpoint the exact source of a problem and determine the most effective intervention. This leads to a "repair-assignment problem" where the same visible failure could require vastly different solutions, such as model retraining, harness engineering, or environment redesign. To overcome this, the researchers propose an interaction-centric taxonomy. This framework localizes failures to the specific interactions where they originate and identifies the responsible component within the agent system, which includes models, harnesses, users, tools, memory, and environments. By categorizing 41 distinct failure modes and assigning them to an interaction edge and a fault side, the taxonomy provides clear guidance for targeted repairs. For instance, "model-side" failures indicate a need for post-training, while "harness-side" failures suggest fixes in scaffolding or tool integration. The schema is designed to be broadly applicable across various agent architectures, from coding assistants to complex multi-agent systems. The study validates its reproducibility using independent reasoning agents, achieving a strong agreement with human annotations.