RxBrain: Embodied AI Model Integrates Language, Vision, Imagination

Embodied cognition requires AI agents to bridge high-level task reasoning with the physical states they need to achieve. This paper introduces RxBrain, an embodied cognition foundation model designed for joint language-visual reasoning and imagination. Unlike traditional vision-language models that focus on scene understanding or generative world models that predict future visual states, RxBrain uniquely represents embodied plans within a single, unified planning sequence. In RxBrain's architecture, language provides the abstract structure of a plan, including task decomposition, constraints, and decision logic, while visual imagination grounds this structure by predicting world states and facilitating joint subgoal planning. Each planning step is associated with intermediate and final physical states. The model employs a unified multimodal Mixture-of-Transformers architecture capable of understanding and generating language, images, and video. To train this capability, an automatic pipeline converts embodied videos into joint text-visual planning supervision. The researchers also developed RxBrain-Bench to evaluate the model's ability to represent embodied plans through coupled textual and visual components. Experiments demonstrate RxBrain's proficiency in embodied understanding and generation, producing plans with integrated textual reasoning, world state prediction, and joint subgoal planning. Furthermore, RxBrain shows promising real-robot performance for continuous action generation, even without extensive action-data pretraining.