New Framework Analyzes Physics-Informed Neural Networks Training Dynamics

The Neural Tangent Kernel (NTK) is a key analytical tool for understanding how over-parameterized neural networks train. This theoretical framework has been extended to Physics-Informed Neural Networks (PINNs), which are popular for solving linear Partial Differential Equations (PDEs). A critical aspect of this analysis for PINNs is establishing the positivity of the associated NTK, a task made difficult by the multiple differential operators involved. To address this challenge, a new theoretical framework called the Differential Neural Tangent Kernel (DNTK) has been proposed. This framework allows for the analysis of PINNs through the NTK lens. The researchers successfully established the positivity of the infinite-width DNTK for both shallow and deep neural networks, covering a broad range of activation functions, including RePU and smooth non-polynomial types, and for all linear differential operators. These significant theoretical results lay a crucial groundwork for the future analysis and development of gradient-based algorithms used to train PINNs.