Koopman Operators Enhanced with Attention-Free Transformers for Prediction

New research aims to enhance the robustness of Koopman operator-based predictors, which are used for linear prediction in a latent space but often suffer from error accumulation over long horizons. The study proposes two complementary innovations: an attention-free latent memory (AFT) block and dynamic re-encoding. The AFT block efficiently aggregates past latent states to correct predictions, operating in linear time with minimal parameters, thereby suppressing error divergence. The second component, dynamic re-encoding, involves lightweight, online change-point triggers that detect latent drift and project predictions back onto the autoencoder manifold. Across various benchmark systems, the Koopman+AFT model, especially with optional re-encoding, consistently demonstrated significantly reduced long-horizon error accumulation compared to standard Koopman autoencoders and other transformer-based approaches. This results in a fast, compact predictor capable of maintaining accuracy over extended prediction horizons.