PPT-Eval: New Benchmark for AI Agents on PowerPoint Tasks

Creating and editing presentations is a ubiquitous and complex activity in professional settings, making it an ideal testbed for evaluating the capabilities of real-world computer-use agents. To address the lack of standardized evaluation in this domain, a new benchmark called PPT-Eval has been introduced. This benchmark comprises 120 diverse PowerPoint tasks spread across 12 files, covering both content creation and editing scenarios, organized by difficulty levels. A central innovation of PPT-Eval is its robust evaluation framework, which tackles the challenge of assessing complex, multimodal tasks that often have multiple valid solutions. Inspired by previous work, this framework uses task-specific rubrics to award partial credit for intermediate steps, penalize unnecessary changes or poor aesthetics, and provide detailed natural language feedback. This nuanced approach has shown a high correlation with human judgments. Initial evaluations using frontier agents like Claude-4.5-Opus reveal that current models still struggle, achieving only a 45% success rate and an average partial score of 57%, indicating significant room for improvement in AI's ability to handle real-world office automation.