New Training Method Improves Long-Horizon Neural Operator Accuracy

This paper introduces HERO (History-Enriched Rollout Training), a novel approach designed to improve the accuracy and stability of neural operators, particularly for long-horizon predictions in time-dependent partial differential equations (PDEs). Neural operators typically suffer from error accumulation because they recursively feed their own predictions back as input, leading to a mismatch between training and inference conditions. HERO addresses this by augmenting traditional absolute trajectory supervision with a relative supervision mechanism. This mechanism leverages the model's optimization history, ranking candidate rollouts from lagged operators and perturbed inputs to identify and learn from "failure trajectories." This creates a margin-based objective that reweights ground-truth gradients, guiding the model to overcome past long-horizon prediction issues. Evaluations across nine PDE benchmarks demonstrate that HERO consistently enhances long-horizon accuracy, extends stable rollout lengths, and improves out-of-distribution robustness without incurring any additional cost during inference. This method offers a significant step forward in stabilizing autoregressive prediction for complex dynamic systems.