AUTOPILOT-VQA Benchmarks Dashcam Understanding for Autonomous Driving.

A new benchmark dataset, AUTOPILOT-VQA, has been developed to rigorously evaluate Vision-Language Models (VLMs) and Multimodal Large Language Models (MLLMs) for their ability to understand and reason about safety-critical incidents captured in dashcam videos. While AI has advanced autonomous driving tasks like scene understanding, assessing reliability in incident scenarios remains a challenge. AUTOPILOT-VQA addresses this by providing structured questions centered around real-world driving incidents and near-incidents. The benchmark covers a wide array of safety-relevant categories, including environmental conditions (weather, lighting), traffic dynamics, road characteristics, signage, involved entities, accident specifics (occurrence, impact location), and reasoning related to avoidability. By requiring models to answer questions grounded in both broad contextual scene properties and granular event-level details, AUTOPILOT-VQA pushes beyond mere object recognition. It aims to foster the development of more temporally grounded, safety-aware reasoning capabilities in autonomous driving systems. The dataset is part of the AUTOPILOT CVPR 2026 competition, offering a standardized tool to improve the interpretability, robustness, and safety consciousness of VLMs for real-world autonomous driving applications.