MultiView-Bench Evaluates VLM 3D Scene Comprehension Across Views.

This research introduces MultiView-Bench, a novel diagnostic benchmark specifically created to evaluate how well Vision-Language Models (VLMs) can integrate visual information from multiple perspectives to form a consistent, world-centric understanding of a 3D scene. Unlike existing benchmarks that primarily focus on single-view or camera-relative perception, MultiView-Bench challenges models to decouple object positions from transient viewpoints and ground them within a fixed global coordinate system. This capability is crucial for advanced applications like robotic assembly. Evaluations of leading VLMs using MultiView-Bench exposed significant limitations. While models performed well on 2D planar relationships from a single image, they struggled considerably with 3D spatial reasoning and with effectively combining information across different views. The study also identified biases, such as difficulties with unconventional axis directions and sensitivity to object colorways or textures. To address these shortcomings, the researchers propose ViewNavigator, a multi-agent framework that actively selects informative viewpoints, perceives, and fuses multi-view evidence, demonstrating substantial improvements for various base models.