New RL Framework Improves Spatial Reasoning, Controls Response Length

Multi-view spatial reasoning tasks, which require vision-language models to compare visual information across images and infer spatial relationships, often lead to overly verbose chain-of-thought reasoning without a corresponding increase in accuracy. Traditional reinforcement learning approaches, relying on sparse outcome-level feedback, struggle to pinpoint reasoning errors or control response length. To address these challenges, researchers propose LenGuard-GPC, a novel dense reward framework. This framework calculates a token-sum KL divergence by comparing predictive distributions under a standard prompt and a guided prompt, providing a granular reward signal. Furthermore, LenGuard-GPC incorporates a staged length bonus. This mechanism prevents the KL penalty from simply favoring shorter responses, ensuring that reasoning length remains within a controlled, optimal range without sacrificing quality. Experiments on six multi-view spatial reasoning benchmarks show that LenGuard-GPC improves accuracy over vanilla GRPO while simultaneously reducing average response length.