Restricted Visibility Boosts Compositional Generalization in Language Models.

A study investigated how restricting the visibility of evidence to individual modules within multi-module language model systems impacts learning and generalization. Contrary to the common practice of exposing all modules to the full input, this research explored whether limiting information flow could lead to more effective solutions. The experiment involved "four-cell societies" of language models that shared a frozen pretrained model and a low-rank adapter, communicating only through two continuous vectors. On a natural-language function-composition task, societies with restricted evidence visibility consistently outperformed their globally visible counterparts, showing at least a 20-point advantage in most pairs. This suggests that constrained communication fosters a more compositional form of generalization, leading to more reusable and value-indexed interfaces between modules. Although the overall performance did not meet the preregistered floor for a "complete pass," the significant paired advantages highlight that restricted visibility substantially increases the probability of discovering a generalizing relay, even for programs whose composite function was unseen during training.