Local Linear Transformer Boosts PDE Operator Learning Efficiency

A new neural operator, the Local Linear Transformer (LLT), has been developed to improve the learning of Partial Differential Equation (PDE) solution maps and accelerate numerical simulations. Traditional transformer-based neural operators, while good at capturing long-range dependencies, suffer from quadratic scaling with the number of computational nodes and lack an inherent bias towards local interactions, which are crucial in many physical systems. LLT tackles these limitations by integrating linear global attention with local spatial mixing. It also explicitly incorporates coordinate and geometry information, making it more suitable for PDE problems. This architectural design allows LLT to maintain the benefits of attention while significantly improving computational efficiency. Evaluations across diverse PDE problems, including elasticity, plasticity, airfoil flow, pipe flow, and Darcy flow, demonstrated LLT's effectiveness. It achieved competitive or lower relative L2 error compared to other neural operator and transformer baselines. Furthermore, LLT reduced wall-clock time per training iteration by factors of 1.8 to 2.5 compared to Transolver on matched structured discretizations, and successfully scaled to complex 3D aerodynamics datasets.