Readout Feedback Steers Recurrent Models During Inference

Recurrent models are increasingly used for complex reasoning tasks, but existing inference-time methods for scaling computation often involve running more steps or sampling multiple trajectories, which can be inefficient. This paper proposes a novel technique called Readout Feedback (RoFB) that enhances recurrent models by leveraging their own intermediate readout probabilities to guide their internal latent dynamics. RoFB works by converting these intermediate predictions into token-wise pairwise coupling forces, which are then injected back into the model's latent states. Crucially, this steering mechanism operates entirely at inference time, requiring no additional retraining of the model. Experiments across various recurrent models and tasks, including Sudoku and Maze solving, demonstrate that RoFB yields substantial performance gains, often surpassing the results of simply increasing computation steps or trajectory sampling, while maintaining comparable or even lower computational costs.