VAORA Improves VLM Physical Reasoning and Task Generalization

Vision-Language Models (VLMs) often struggle with generalizing their physical reasoning capabilities, particularly when faced with new tasks or environments. Two primary issues are prevalent: the generation of "hallucinated" reasoning chains that contradict physical reality, and a disconnect between the model's internal reasoning and its actual actions. To address these challenges, researchers have introduced VAORA (Visual Action Outcome Reasoning Alignment), a new reward design. VAORA incorporates two distinct rewards: a Visual Alignment Reward, which grounds VLM reasoning in the visual context independently of the agent's action, and a Visual-Action Alignment Reward, which ties reasoning to the visual outcome produced by the model's actions. These combined rewards work to suppress physically inconsistent reasoning and bridge the gap between a model's thought process and its behavior. The framework also utilizes smooth, dense rewards derived from an expert agent's success probabilities to enhance training stability. Experiments on PHYRE and Virtual Tool benchmarks confirm VAORA's effectiveness in improving VLM performance across novel tasks and unseen environments.