Memory Architecture Crucial for Language Emergence in LLM Agents

Researchers explored how two large language model (LLM) agents can spontaneously create a shared language in a Lewis signaling game, where they must coordinate a code based solely on their interaction history. The study compared five different memory architectures across various communication channel configurations. A key finding was that the agents' memory architecture played a more critical role in language emergence and coordination than the raw capacity of the communication channel itself. Specifically, agents equipped with a persistent private notebook demonstrated the most reliable coordination, even benefiting from surplus channel capacity. This "notebook" allowed agents to externalize and retain learned conventions, preventing them from having to re-derive communication codes in each round. In contrast, stateless agents struggled as vocabulary grew, with their performance degrading when the context window couldn't track all necessary information. The research suggests that understanding how memory architecture enables agents to transform interaction history into stable conventions is essential for comprehending language emergence.