Rank-Order Encoding Boosts Sparse Distributed Memory Robustness

Large language models often struggle with continual learning, prompting renewed interest in Sparse Distributed Memory (SDM) as a potential solution for explicit online episodic memory. This research explores the effectiveness of rank-order N-of-M encoding as an alternative to the traditional threshold-binary encoder in SDM architectures. The study first validates a faithful reimplementation of the architecture, confirming its equivalence and reproducing known behaviors under interference. Capacity experiments then demonstrate that RankOrderSDM significantly outperforms StandardSDM in terms of memory capacity, particularly at saturation. Crucially, the research disentangles the contributions of the encoding scheme and the learning rule to noise robustness. It reveals that the substantial gain in robustness primarily stems from the synergistic interaction between rank-order encoding and MAX-Hebbian learning, rather than the encoder alone. These findings provide practical insights for designing more robust memory-augmented AI systems, such as those inspired by CALM.