ImagingBench Reveals AI Weakness in Computational Imaging

While vision-language models (VLMs) and agentic AI have demonstrated strong performance in semantic visual tasks, their ability to handle the underlying physics and inverse problems of computational imaging has been unclear. To address this, a new benchmark called ImagingBench has been introduced. ImagingBench comprises 20 computational imaging tasks categorized into ray/wave optics, image signal processing, inverse reconstruction, computational sensing, and calibration. It evaluates models in three settings: Expert (fixed expert-guided reconstruction), Planner (planner-guided reconstruction), and Forward (forward-system simulation for consistency). Leading proprietary and open-source multimodal systems, including Gemini, GPT, and Qwen, were benchmarked against specialized non-agentic baselines. The results consistently show that agentic models are significantly weaker than specialized methods, especially in complex computational sensing problems like lensless imaging or time-of-flight imaging. Even with planner guidance, gains were modest and inconsistent. Although the AI models often produce visually convincing images, their fidelity when compared to reference solutions remains poor, indicating a substantial gap between their semantic understanding of images and their grasp of the physical principles governing imaging processes.