SAAG Improves Diagnostic Evaluation for LLM Agent Calling

A new diagnostic framework, SAAG (Structured Agent Assessment and Grounding), has been introduced to provide a more nuanced evaluation of how large language models (LLMs) interact with and call external agents or tools. Traditional evaluation methods often use a single binary score, which obscures the specific reasons behind agent-calling failures, making it difficult for developers to diagnose and fix issues. SAAG addresses this by breaking down the agent-calling process into three sequential stages: checking if the model conforms to the agent registry, ensuring the structural completeness of the call, and verifying the grounding of arguments. Each stage produces specific, interpretable diagnostics. This structured feedback not only helps in understanding where failures occur but also enables iterative self-repair, guiding targeted corrections without revealing ground-truth values. Experiments show that this stage-decomposed evaluation consistently improves argument precision and reduces hallucination in argument values, offering a clearer lens for improving agent-calling reliability.