Action QFormer Improves VLA Model Performance and Stability

In Vision-Language-Action (VLA) models, action supervision is typically seen as a downstream goal for predicting actions. However, this research explores action supervision as a force that actively shapes the multimodal representations within the model. A key challenge arises because directly applying action supervision can destabilize representations crucial for language understanding and object grounding, even as it helps form action-compatible ones. To resolve this tension, the researchers developed Action QFormer, a novel query-based, action-facing interface. This interface uses instruction-conditioned queries to intelligently reorganize inherited multimodal information into representations optimized for action generation, before the final action is produced. In zero-shot sim-to-real navigation tasks, Action QFormer significantly boosted average closed-loop task success and action-generation correctness, while nearly eliminating out-of-distribution instruction generations. The analysis shows it refines how action supervision impacts representations, allowing for targeted adaptation without broad upstream disruption.