BRAID Unifies Multi-Modal Reasoning with RL

Unified multi-modal models (UMMs) have demonstrated impressive capabilities in interleaved text-image reasoning, but optimizing their multi-turn generation through reinforcement learning (RL) has remained a significant challenge. Existing RL approaches typically apply optimization only to text generation steps, treating image generation as a separate, supervised process. This prevents the full propagation of policy gradients across the entire interleaved trajectory, leaving much of RL's potential for UMMs untapped. This research introduces BRAID (Bridging Interleaved Multi-modal Reasoning as a Unified Decision Process), a straightforward yet powerful framework. BRAID reframes multi-turn text-image-text reasoning as a unified Markov Decision Process (MDP), allowing for the joint optimization of both textual and visual generation using a single, coherent RL objective. BRAID achieves this by computing a shared trajectory-level advantage, which is then consistently propagated into both text tokens and image denoising paths, each optimized through its native policy gradient mechanism. To further enhance learning over long horizons, BRAID incorporates a vision-language model (VLM) judge that scores each intermediate image based on its reasoning utility, providing dense, turn-level feedback to sharpen learning at critical visual decision points. Experiments on spatial reasoning and visual perception benchmarks confirm that BRAID consistently outperforms various baselines, validating the effectiveness of its unified MDP formulation and vision-thinking guidance for multi-modal reasoning.