Spectral Identifiability for Mechanistic Interpretability Proposed

Mechanistic interpretability aims to understand how neural networks function by identifying internal "circuits" or features. A significant challenge in this field is determining whether an identified circuit is a genuine property of the model or merely an artifact of the specific interpretability method used. For instance, sparse autoencoders can yield different features from the same activations depending on initialization or architecture, lacking a theoretical basis to distinguish structural properties from incidental variations. This paper tackles this problem by placing dictionary learning for interpretability on a rigorous "identifiability" footing. It proposes treating the forward pass of a neural network as a controlled dynamical system, with depth acting as time. By lifting this system with the Koopman operator, a finite linear realization is obtained whose *spectrum* is a coordinate-free, intrinsic property of the model itself. The researchers prove that this spectrum is recoverable from calibration samples with a specific error rate, establishing the first identifiability theorem for a mechanistic interpretability primitive. They also provide a matching minimax lower bound and a variant for heavy-tailed activations. A key finding is a "dissociation theorem," which states that when the realization is non-normal, the directions carrying activation variance and those carrying information across depth cannot coincide, implying that the identifiable object and the legible object are not always the same. Experiments on GPT-2 small, Gemma-2-2B, and Qwen3-8B-Base confirm the spectrum's convergence and predicted exponent, showing that Koopman modes offer insights into model structure, albeit not always outperforming principal components for direct object identification.