Spectral Analysis Detects Neural Network Failures

Neural networks often fail silently, with misclassifications not always evident from their output confidence scores. This study identifies a new phenomenon called "Spectral Drift," which describes characteristic spectral instabilities within a network's internal activations when it makes an error. This drift is not visible at the final output layer, explaining why traditional confidence-based error detection methods are often ineffective. The researchers propose Self-Detecting Neural Networks (SDNN), a novel framework that monitors these spectral dynamics across the network's layers. SDNN employs techniques like Short-Time Fourier Transform and wavelet decomposition to extract multi-scale spectral features. A small, dedicated detector network, trained with curriculum learning on various failure types, then learns to identify these failure-indicative patterns. Experiments on CIFAR-10 show that SDNN significantly outperforms existing confidence-based baselines in detecting misclassifications, distribution shifts, and adversarial attacks, demonstrating a substantial improvement in reliability.