Epistemic Working Memory Boosts Multi-Hop Reasoning in Language Agents

Language agents often falter when tasked with multi-hop reasoning, even if individual steps are simple. This degradation is attributed to "context dilution," where crucial early information becomes obscured within an expanding context window. To address this, researchers developed SLEUTH, a framework that equips language agents with a structured epistemic working memory. This memory explicitly organizes the agent's investigative state into three categories: "Confirmed Facts" linked to their sources, "Active Hypotheses" ranked by supporting evidence, and "Open Questions" that directly guide subsequent actions. SLEUTH was tested across five multi-hop benchmarks and compared against five established baselines. Its performance advantage grew with problem difficulty, showing improvements of up to +11 points on 4-hop chains and surpassing methods like Reflexion without needing multiple episodes. Further analysis revealed an "evidence sufficiency problem," where agents often find the answer but fail to commit, wasting resources on unnecessary verification. A lightweight commitment trigger, when combined with SLEUTH's structured state, resolved this issue, yielding significant gains (up to +19 points on hard problems). This highlights that an organized epistemic state, rather than just raw model capability, is the key factor for scaling complex multi-hop reasoning, even enabling weaker models to perform better through protocol adherence.