New Framework Reduces MLLM Error Snowballing in Reasoning Traces

Autoregressive multimodal large language models (MLLMs) often suffer from "error snowballing," where an initial incorrect inference in a chain-of-thought (CoT) trace leads to cascading failures. Current state-of-the-art open-source MLLMs exhibit a 65% snowball rate once an error occurs. Existing mitigation strategies, such as multiple sampling or post-hoc verification, are either too late, lack symbolic grounding, or sacrifice the flexibility of natural language. Researchers propose Constraint-Anchored Reasoning Traces (CART), a novel neuro-symbolic framework. CART trains MLLMs to embed symbolic constraint assertions—lightweight, verifiable statements about visual content—within their natural language reasoning steps. A Constraint Propagation Module continuously verifies these anchors against visual features and checks their logical consistency. Upon detecting a contradiction, a backtrack controller reverts the generation to the last consistent state, effectively preventing error propagation. This adaptive mechanism allows models to control anchor density, avoiding unnecessary trace bloat. Evaluations across five benchmarks, using fine-tuned LLaVA-NeXT and Qwen2-VL models, show CART reducing the snowball rate from 0.65 to 0.14. It also improves GQA accuracy by 4.6 percentage points and achieves 89.1 F1 on POPE-all with minimal inference overhead. This demonstrates a significant step towards more reliable and robust MLLM reasoning.