Spectral Outliers Uncover Transformer Attention Structure

This research delves into the internal workings of transformer attention mechanisms by applying Marchenko-Pastur (MP) random matrix theory. The goal was to decompose the attention projection matrices into a "random-like bulk" and distinct "spectral outliers." This method aims to differentiate between noise and meaningful learned structure within the model. The study causally validated this decomposition by testing its impact on model performance. When the MP-identified spectral outliers were zeroed out in Mistral-7B, the model's performance on benchmarks like HellaSwag, MMLU, and PIQA plummeted to near random-chance levels. In contrast, zeroing an equivalent number of singular values from the "bulk" caused only minor degradation, confirming that these outliers represent the dominant learned signal. Across 11 different pre-trained transformers, five consistent patterns emerged. Spectral outliers consistently encoded the most significant learned structure, with Query (Q) projections exhibiting the highest number of outliers. The Value (V) projections in grouped-query attention, however, showed less clear signal-to-noise separation. Furthermore, specific entry-level outliers formed structured patterns in Q and Output (O) matrices, and certain residual-stream dimensions consistently appeared as band outliers across layers in Key (K) and O matrices. These insights offer a foundation for developing more parameter-efficient fine-tuning and structured pruning techniques for transformers.