New Framework Enhances Reasoning in Dense Retrieval Systems

Traditional reasoning-enhanced embedding models for dense retrieval often focus on the final retrieval outcome, potentially leading to shortcut reasoning patterns that don't fully leverage intermediate reasoning steps. A new framework, Retrieval Grounding Latent Reasoning (RGLT), aims to address this by making the connection between latent reasoning trajectories and retrieval improvements more explicit. RGLT operates by performing non-autoregressive reasoning within a hidden space, constructing an instruction-conditioned latent reasoning path using silent tokens. It integrates two key supervisory mechanisms: process-supervised explicit-to-implicit distillation, which shapes intermediate latent states through stage-wise Chain-of-Thought (CoT) reconstruction, and retrieval-grounded supervision, which optimizes incremental retrieval gains across these latent reasoning trajectories. This dual approach ensures that the reasoning process is both meaningful and directly contributes to better retrieval outcomes.