Spatial Memory Agent Boosts VLM Spatial Reasoning Without Updates

Improving the spatial reasoning capabilities of Vision-Language Models (VLMs) is crucial for embodied agents and multimodal assistants. Current approaches typically involve post-training methods like fine-tuning or integrating external spatial tools. This research explores a novel, complementary path: enhancing frozen VLM agents through "parameter-update-free self-evolution." The proposed Spatial Memory Agent (SMA) is an experience-grounded runtime framework. It operates by allowing a frozen VLM agent to query a verifiable spatial environment, receive feedback, and then use verifier-guided reflection to distill compact, transferable "lessons" from these experiences. Each lesson is assigned a Transfer Reliability Score (TRS), which is dynamically calibrated based on its effectiveness in future retrievals. During deployment, SMA retrieves relevant lessons using a combination of semantic filtering and TRS-based ranking, which then guides the frozen VLM's inference. This method consistently outperforms other evaluated approaches across five spatial benchmarks and four base VLMs, demonstrating a practical way for VLMs to self-evolve their spatial intelligence without requiring parameter updates.