LongWebBench Evaluates Long-Horizon Webpage Generation by VLMs

A new benchmark, LongWebBench, has been introduced to rigorously assess the capabilities of Vision-Language Models (VLMs) in generating webpages. Unlike previous evaluations that focused on short, static pages, LongWebBench targets long-horizon webpages, considering both their structural integrity and functional interactivity. The benchmark comprises a dataset of 490 real-world long webpages for structural evaluation and 507 interactive tasks across 129 webpages to test functional performance. It employs a multi-dimensional VLM-based metric for structural coherence and an agent-based pipeline for end-to-end functional verification. Initial experiments with state-of-the-art VLMs reveal that structural fidelity degrades with increasing webpage length, and visually plausible generations often fail to support multi-step interactions. This highlights the necessity of evaluating webpage generation beyond mere visual similarity, emphasizing executable interaction as a core criterion.