Single Neuron Edits Can Mitigate LLM Repetition Loops

A study explored the possibility of resolving persistent repetition loops in large language models, specifically the Gemma 4 instruction-tuned models, through targeted weight edits. These models frequently exhibit repetitive behavior when asked to enumerate long lists of facts, a problem that resists common prompting or inference adjustments. The researchers localized the cause of these loops to a small number of MLP neurons or, in larger Mixture-of-Experts models, specific routed experts. By applying static weight edits, sometimes as minimal as inverting the sign of a single neuron, they successfully suppressed these repetition patterns. This intervention maintained general benchmark scores while significantly reducing looping. However, the study also distinguished between simple repetition loops and more complex "doom loops," where models endlessly self-correct without converging on an answer. While the neuron edits reduced doom looping, they did not eliminate it, suggesting that this deeper issue stems from a lack of precise knowledge rather than a removable circuit. The findings demonstrate the feasibility of localizing and editing specific generation pathologies but also highlight the limits of such an approach for knowledge-based deficiencies.