Local Linear Transformer Accelerates PDE Operator Learning.

A new neural operator architecture, the Local Linear Transformer (LLT), has been developed to improve the learning of Partial Differential Equation (PDE) solution maps and accelerate numerical simulations. While transformer-based neural operators are valuable for capturing long-range dependencies, they typically suffer from quadratic scaling with computational nodes and lack an inherent bias towards local interactions, which are crucial in many physical systems. LLT addresses these limitations by integrating linear global attention with explicit local spatial mixing. It also incorporates coordinate and geometry information, making it well-suited for diverse PDE problems. This design allows LLT to maintain the benefits of attention for long-range dependencies while being computationally more efficient and physically informed. The model was evaluated across various PDE problems, including elasticity, plasticity, and fluid flows, using data from different discretization methods and mesh types. LLT consistently achieved competitive or lower relative L2 error compared to other neural operator and transformer baselines. Furthermore, it demonstrated significant computational efficiency, reducing wall-clock training time by factors of 1.8 to 2.5 compared to Transolver on structured discretizations, and successfully scaled to complex 3D aerodynamics datasets.