New Method Detects "Dead Directions" in Neural Networks

Understanding the internal structure and behavior of trained neural networks is crucial for improving their performance and interpretability. A new method has been introduced that allows for the measurement of "singular structure," often referred to as "dead directions," within these networks. This technique is notable because it operates without requiring gradient descent or canonical alignment, making it highly versatile and applicable to frozen checkpoints of trained models. The core of this method involves recovering the order k of each dead direction directly from the directional-Fisher rate. This rate serves as a master invariant, from which the per-direction learning coefficient can be precisely derived, regardless of the optimizer's basis. Furthermore, the method can classify each identified direction, differentiating between a genuine singularity, whose order is fixed by the network's architecture, and a flat gauge symmetry. The magnitude of the directional-Fisher helps resolve ambiguous cases. A key feature is a pluggable detector compatible with various common neural network components, including transformer, convolutional, and normalization layers. The research successfully demonstrated that this method accurately recovers architecture-predicted orders across both constructed cells and fully trained networks. Examples include identifying the LayerNorm-kernel gauge in a fine-tuned vision transformer and a node-death in the compressed MLP of a from-scratch transformer. This approach effectively transforms order-recovery into a deterministic, architecture-general reading, providing deep insights into network singularities.