Predicting LLM Compositional Failures Using Feature Geometry.

Identifying challenging scenarios for Large Language Models (LLMs) typically involves human intuition or extensive benchmark curation. This paper introduces a novel approach: predicting an LLM's compositional errors by analyzing its internal representational geometry. The core finding is that compositional failure can be attributed to interference between salient features within the model's internal representations. Specifically, if a pair of concepts is encoded with near-orthogonal linear representations, the LLM reliably composes them. Conversely, if their linear encodings are too close, leading to interference, the model struggles to compose them effectively. This method reliably anticipates failure modes across various compositional tasks, including programmatic settings, multi-hop reasoning, and multilingual factual recall, without requiring evaluation of specific inputs. These results provide a scalable foundation for identifying high-risk examples, constructing targeted stress tests, and enhancing active learning strategies in real-world LLM deployments.