Neural Networks Learn Self-Knowledge Through Self-Interventional Learning

Machine learning systems typically operate by modeling external data, with their internal workings analyzed by external observers. This new work proposes Self-Interventional Learning (SIL), an innovative approach where a neural network actively modifies its own functional architecture. By performing these internal perturbations and observing the resulting changes, the network constructs a predictive model of its own behavior. This self-model allows the system to generalize to interventions it hasn't explicitly executed and to use these predictions to inform subsequent structural adjustments. In synthetic systems, SIL successfully identified crucial structural elements, redundancies, and replaceability within the network. While the method showed significant improvements in predicting intervention outcomes with increased budgets, its model-guided actions did not consistently outperform simpler direct empirical strategies, and it offered no robustness advantage in a CIFAR-10/ResNet validation. The findings confirm SIL as a viable framework for neural networks to acquire predictive knowledge about their own functional organization through self-experimentation. However, they also highlight that this self-model remains incomplete and is not universally superior to more straightforward approaches in all scenarios.