Multi-Agent AI Pipeline Enhances Auditable Financial Chart Analysis

This research introduces AgentFinVQA, a novel multi-agent system specifically engineered for analyzing financial charts and answering related questions. The system is designed with a strong emphasis on auditability and the ability to be deployed on-premise, addressing critical requirements in regulated financial environments. It achieves this by breaking down complex queries into distinct stages such as planning, optical character recognition (OCR), legend interpretation, visual inspection, and verification, meticulously documenting each step in a Model Evaluation Packet. AgentFinVQA demonstrates significant performance improvements, outperforming proprietary zero-shot baselines on the FinMME benchmark. Notably, it maintains strong accuracy even when utilizing open-weight models served locally, which is crucial for institutions that cannot transmit sensitive client data to external providers. The system's built-in verifier also provides a confidence signal, enabling human-in-the-loop review for potentially uncertain answers. Analysis of errors points to common issues like question misinterpretation and legend confusion, offering clear directions for future development. The project emphasizes that auditable and on-premise financial chart question answering is now practical, with open-weight solutions retaining most of the accuracy benefits while ensuring full data residency.