GUI Agents Outperform CLI, Skill Augmentation Boosts CLI

Computer-use agents can interact with software either through graphical user interfaces (GUI) or command-line interfaces (CLI). Existing evaluations often conflate interaction modality with other variables, making direct comparisons difficult. This research introduces a standardized benchmark of 440 desktop tasks across various applications and workflows, ensuring identical goals and states for both GUI and CLI agents, with actions restricted to their native modalities. In this controlled environment, the most effective screen-only GUI agent achieved a 59.1% success rate, surpassing the strongest original-skill CLI agent, which reached 48.2%. However, when the CLI agent's skills were augmented with verifier-guided enhancements, its success rate climbed to 69.3%. This indicates that the initial performance gap for CLI agents was largely due to incomplete skill coverage rather than inherent model limitations. The findings suggest that GUI agents face challenges with reliable grounded interaction over extended, complex workflows. Conversely, CLI agents are primarily constrained by the breadth and scalability of their skill interfaces. This distinction highlights different bottlenecks for each interaction paradigm, offering insights for future development of more capable computer-use agents.