MLLMs Learn Rule-Compliant Visual Spatial Planning with New Benchmark

Multimodal Large Language Models (MLLMs), which combine visual perception with linguistic reasoning, have a critical gap in their ability to perform visual spatial planning while adhering to explicit or novel rule constraints. To address this, a new benchmark called RuleMaze has been developed. RuleMaze challenges MLLMs to navigate mazes by interpreting natural-language rules of varying complexity, thereby isolating the specific skill of rule-compliant spatial planning. To facilitate scalable rule construction, the researchers developed Language-Logic-Function Hybridization, an automated system that generates natural-language rules and translates them into logical representations and executable validators. This eliminates the need for manual rule engineering. Furthermore, they introduced Disentangled Multimodal Planning (DMP), an innovative approach that explicitly separates the MLLM's perception, action execution, and rule verification processes. By disentangling these components, DMP significantly enhances rule following and generalization to new, more complex rules, while also providing transparent intermediate planning traces, outperforming traditional end-to-end textual planning methods.