Epistemic Priors Enable Sophisticated Closed-Loop AI Planning

This paper re-evaluates Sophisticated Inference within active inference, arguing that its primary role is to establish closed-loop control within a planning horizon. This means future actions can dynamically adapt based on anticipated future states and observations. The authors propose that this closed-loop structure can be effectively represented using the epistemic-prior variational free energy framework, where epistemic priors guide the active inference objective and a joint posterior over future states and actions defines the state-contingent control. The study validates this decomposition using the Reactivity Maze, a benchmark designed to isolate epistemic incentives from inner-horizon closed-loop control. Comparing various variational objectives, including action-state factorized active inference and standard Expected Free Energy planning, the results indicate that neither an epistemic component nor the ability for future actions to depend on future states is sufficient on its own. Only methods that integrate both—like Sophisticated Inference and full-joint epistemic-prior active inference—successfully solve the environment by combining information-seeking behavior with reliable goal-reaching capabilities. This suggests the advantage of Sophisticated Inference stems from its closed-loop active inference form, rather than being exclusive to tree search.