New Benchmark Diagnoses AI Agent Failures in Web UI Interactions

Current methods for evaluating AI agents that interact with computers often focus on either very long, complex workflows or extremely basic GUI tests. This leaves a gap in understanding how agents perform on realistic, component-specific interactions that are short enough for diagnosis but rich enough to reflect modern interface challenges. To address this, researchers introduced ComponentBench, a new benchmark and diagnostic system. It features an ontology of 97 common UI components, translated into 2,910 verifiable tasks across various component libraries. The benchmark also includes human reference trajectories to assess both task success and interaction efficiency. Evaluations of seven prominent AI models, including GPT-5.4 and Gemini 3 Flash, demonstrated that the choice of observation and action space significantly affects performance. For instance, GPT-5 mini's success rate dropped from 83.1% with accessibility-tree observations to 48.9% with pixel-only control. Even the fastest AI configurations were significantly slower than human performance, and simple spatial tasks continue to pose difficulties for current agents.