New Method Extracts Interpretable Mechanisms Directly from LLM Weights

Current approaches to mechanistic interpretability often rely on training proxy dictionaries, such as sparse autoencoders, to label features based on max-activating text. While these methods can identify concepts, the interpretability resides within the learned dictionary rather than the intrinsic network weights. This new research proposes an alternative: Tiled SVD.Tiled SVD directly extracts "mechanism mounts" from linear sites within a model's weights. Each mount is defined as a triple: a trigger vector, a write vector, and a strength scalar, which together represent a specific weight rule. This allows for direct identification of how the network's weights contribute to its operations.The method was evaluated on Gemma-2-2B using a WikiText-2 subsample, scoring all seven linear maps. The results indicate high success rates, particularly for residual writes (mlp.down, attn.o), which passed all 52 site-layers. Other maps also showed strong performance. The authors have released the library code and experimental tools for reproducibility.