RobustMAD Benchmark Evaluates Small Multimodal LLM Robustness

Multimodal industrial anomaly inspection assistants are vital for smart factories, but large language models are often too computationally demanding or pose privacy risks for on-site deployment. Compact multimodal small language models (MSLMs) offer a deployable alternative, yet their real-world robustness has been poorly understood due to a lack of comprehensive benchmarks. To address this, researchers developed RobustMAD, the first benchmark specifically designed to evaluate MSLM robustness under diverse, open-ended industrial queries. This benchmark covers object understanding, anomaly detection, unanswerable problems, and visual quality degradations, reflecting realistic industrial conditions. Surprisingly, top-performing MSLMs showed promising capabilities, even outperforming larger models like GPT-5 Nano in some aspects. However, RobustMAD also exposed significant robustness gaps, identifying three recurring failure modes: fragile multimodal grounding under fine-grained distinctions or degraded visuals, insufficiently comprehensive responses, and weak logical grounding leading to hallucinations on ill-posed queries. These insights provide actionable guidance for designing more robust next-generation industrial inspection assistants.