DashArena Benchmarks LLMs for Interactive Dashboard Generation

Evaluating the effectiveness of large language models (LLMs) in generating interactive analytic dashboards has been challenging, as existing methods often fail to capture the full scope of analytical support and interaction quality. A new benchmark, DashArena, addresses this by requiring LLMs to generate both a dashboard and a replayable interaction trajectory. DashArena employs a browser executor to replay these trajectories, providing reproducible visual and execution evidence of the LLM's intended analytical workflow. A Vision-Language Model (VLM) judge then compares candidates based on this evidence, with results aggregated using Bradley-Terry. The research also introduces DashJudge-8B, an open-weight distilled judge that effectively reproduces human judgments. Experiments with current frontier models highlight persistent issues in rendering, analytical accuracy, and interaction capabilities, underscoring that realistic, interactive dashboard generation remains a significant challenge for LLMs.