Model Memory Rewriting During Thought Impairs Learning Speed

The "Memoir" model architecture integrates several components: a per-sample fast memory, shared slow parameters, variable-depth latent recurrence, and a future-latent energy objective. This research specifically investigates a critical design choice within Memoir: whether a model should be allowed to rewrite its fast memory during the same "pondering iteration" in which it reads from it. This is termed the "coupled" approach. The study compared this coupled arm against an otherwise identical "read-only" pondering arm on a procedural associative recall task involving key interference. Both arms were matched in terms of parameters, data, optimizer, schedule, and seeds. After 240 training steps across 12 seeds, the coupled arm achieved a recall of 0.5203, significantly lower than the read-only arm's recall of 0.6557. The read-only arm outperformed the coupled arm in 10 out of 12 seeds, indicating a clear learning-speed penalty for the coupled approach. While both arms eventually reached 1.0000 recall after 960 steps (suggesting the effect is on learning speed rather than ultimate capability), the task became ceiling-limited, preventing measurement of convergence on a non-saturating task. Interestingly, a predicted failure where memory rewriting would corrupt the energy signal did not materialize; the energy margin actually grew and held. The research also noted a significant reduction in delta-rule forward time due to kernel restructuring.