CaVe-VLM-CoT Enhances VLM Interpretability and Reduces Hallucinations

Vision-Language Models (VLMs) frequently produce "hallucinations"—fluent but factually incorrect or visually unfaithful outputs. Existing methods like chain-of-thought and retrieval-augmented generation (RAG) only partially mitigate this, often lacking step-level citation grounding or mechanisms to correct verification failures. A new framework, CaVe-VLM-CoT, addresses these limitations with a modular, reflection-based agentic-RAG approach. It employs a five-stage closed-loop pipeline: Extractor, Retriever, Solver, Citation Injector, and Verifier. Crucially, if the Verifier detects ungrounded claims, it triggers structured feedback to the Extractor for targeted re-retrieval, creating a self-correcting loop. To comprehensively evaluate this framework, the authors also introduce a suite of 23 component-wise metrics, culminating in CaVeScore. This composite metric weights accuracy, citation precision and recall, attribution, and evidence grounding, providing a robust measure of VLM performance and interpretability. The framework demonstrates strong accuracy and CaVeScore on challenging benchmarks without architectural modifications.