LLM Agents Create Shared Lexicons for Unknown Environments

Autonomous agents operating in unexplored environments, such as during planetary or deep-sea exploration, face the challenge of referring to entities that lack existing names in human language. To overcome this, they need to establish shared vocabularies. The Neuro-Symbolic Lexical Discovery (NSLD) framework proposes a solution where a group of LLM-based agents collaboratively develops a shared "alien lexicon." Each agent is equipped with a frozen CLIP vision encoder, a private FAISS vector index, and a text-only LLM. These agents engage in a referential game over out-of-distribution visual referents, autonomously converging on a common vocabulary. A key aspect of NSLD is that the newly discovered alien words are semantically anchored to natural language through their proximity in the embedding space, effectively expanding human vocabulary with perceptually grounded terms. Simulations with up to twenty agents and ten visual referents demonstrated successful consensus, with the convergence dynamics accurately modeled, paving the way for pre-deployment planning in future autonomous exploration missions.