Fourier Differentiation Speeds Up Physics-Informed Neural Networks.

Researchers conducted a controlled comparison between two methods for calculating spatial derivatives in periodic Physics-Informed Neural Networks (PINNs): spatial automatic differentiation (AD) and Fourier spectral differentiation. PINNs often rely on AD, which can become computationally expensive and memory-intensive, especially when dealing with multiple or high-order derivatives required for solving partial differential equations. The study aimed to isolate the impact of the differentiation procedure by keeping other network parameters, optimizers, and training schedules constant. The Fourier variant involves evaluating network outputs on a uniform periodic grid, transforming them into Fourier space, and then obtaining spatial derivatives via spectral multiplication. This approach allows for the reuse of Fourier coefficients across different derivative orders. Across five different equation-framework settings, including Allen-Cahn and Korteweg-de Vries equations, Fourier differentiation consistently yielded substantial training speedups, ranging from 2.90x to 18.52x. Additionally, it reduced peak GPU memory allocation by 68.7% to 94.1%. Crucially, the final solution errors remained comparable between both methods, indicating no significant accuracy trade-off.