Phi-Nav Improves Vision-Language Navigation with Hindsight Instructions.

Training robust Vision-Language Navigation (VLN) agents often relies on on-policy exploration, which exposes the agent to a wider range of states. However, this exploration frequently leads to trajectories that diverge from expert demonstrations, creating a semantic mismatch between the visual observations and the original language instructions. This paper introduces Phi-Nav, a unified on-policy framework designed to bridge this critical semantic supervision gap. Phi-Nav operates through a three-stage dual-supervision cycle. First, the agent explores its environment with oracle guidance, learning from expert actions. Second, a hindsight speaker synthesizes a new, path-level instruction that accurately describes the agent's actual exploratory journey based on collected visual observations. Finally, the agent performs a second imitation pass, treating this synthesized trajectory-instruction pair as an additional expert demonstration. This process effectively converts semantically unlabeled movements into valuable training signals. Evaluations on R2R-CE and RxR-CE benchmarks show that Phi-Nav achieves competitive performance while requiring significantly fewer expert demonstrations than current baseline methods, highlighting the importance of semantic exploration in VLN.