iFLYTEK-Embodied-Omni: Unified Multimodal Model for Embodied Agents

Developing general-purpose embodied AI agents capable of understanding diverse instructions, predicting environmental changes, and executing precise actions over long periods is a significant challenge. Existing approaches often specialize in individual modalities or rely on cascaded pipelines that can introduce errors. iFLYTEK-Embodied-Omni addresses this by introducing a unified multimodal foundation model. This model integrates vision (videos and images), language, and action within a single "Omni" framework. It employs a "brain-cerebellum" collaboration: a high-level vision-language and video generation model handles instruction understanding, task planning, and future state prediction, while a low-level action generation model directly translates planned subgoals and shared context into executable actions. Trained on a comprehensive dataset combining human and robot demonstrations with general image-text data, and using a progressive four-stage training strategy, this model aims to overcome the limitations of specialized or cascaded systems, offering a more robust and integrated approach to embodied AI.