Hybrid Mamba Model Boosts Audio-Visual Navigation

For over five years, the fundamental backbone networks in audio-visual navigation have remained largely unchanged, relying on convolutional neural networks and recurrent architectures. This stagnation has led to inefficiencies in representing dynamic multimodal sequences, limiting progress in the field. Researchers propose Samba, a novel hybrid Mamba model designed to modernize these architectures. Samba replaces traditional GRUs with an adaptive selection-enabled Mamba State Encoder (M-SE) for temporal aggregation. Additionally, it introduces an Audio Mamba Encoder (AME) to overcome the limitations of convolutional operators in capturing global time-frequency dependencies within spectrograms. Experiments on the Matterport3D and Replica datasets demonstrate Samba's exceptional generalization capabilities, particularly with unheard sound sources and unseen scenes. It achieves significant improvements in navigation success rates, while also offering lower computational costs, paving the way for more robust and efficient embodied AI systems.