New Benchmark Evaluates AI on Complex Construction Drawings

Construction drawings are a critical medium in engineering, blending complex visual and textual information like abstract geometry, symbolic notation, tabular data, and specialized annotations. Current multimodal large language models (MLLMs) struggle with this unique domain, as existing benchmarks primarily focus on natural images or simpler floor plans. To bridge this gap, researchers have developed DrawingVQA, a new benchmark specifically designed for real-world construction drawings. It comprises 33 "Issued for Construction" drawings and 92 expertly crafted question-answer pairs, categorized into three levels of reasoning depth: perceptual understanding, contextual interpretation, and domain-expert reasoning. Evaluations of state-of-the-art MLLMs using DrawingVQA reveal a substantial disparity between AI performance and human expert capabilities, particularly for higher-level reasoning tasks. This benchmark is expected to drive advancements in domain-specialized multimodal reasoning, facilitating better integration of AI into engineering workflows.