Energy-Guided Recursive Models Enhance Structured Problem Solving

Recursive reasoning models are designed to tackle structured problems by iteratively updating the latent states of small neural networks. A significant challenge in these models has been the lack of a clear, principled mechanism for inference at test-time; simply increasing depth or breadth generates more potential solutions without a robust way to select the best one. Current methods often rely on auxiliary "q-heads" or heuristic voting, which can be suboptimal. This research introduces the Energy-guided Recursive Model (ERM), which addresses this limitation by incorporating an intrinsic selection principle rooted in explicit Hopfield energies. ERM leverages Hopfield-type memories, which store valid local or global structures, to define a selector over the various candidate trajectories generated by the recursive process. This energy function seamlessly integrates with established energy-based techniques, such as parallel tempering, to significantly boost both sampling efficiency and the accuracy of solution ranking. Evaluations on classic structured problems like Sudoku, Pencil Puzzle Bench (PPBench), and Mazes demonstrate ERM's superior performance. With a modest number of recurrent steps and candidates, ERM achieves near-optimal solutions, substantially improving upon recent probabilistic and equilibrium reasoners. These results highlight that embedding explicit energy functions into recursive reasoning provides a more principled and effective pathway to solving complex structured problems.