EMAGN Boosts Scalability and Efficiency for Traffic Forecasting Models

Traffic forecasting is a complex challenge due to the intricate and dynamic spatial and temporal dependencies involved. While self-attention mechanisms have proven highly effective in modeling these dependencies and achieving state-of-the-art performance, their quadratic computational and memory complexity limits their scalability for large networks. To address this, researchers have developed the Efficient Multi-Attention Graph Network (EMAGN). EMAGN linearizes the spatial attention mechanism by introducing two learned clustering matrices. These matrices adaptively group key and value vectors into "super-clusters," effectively reducing the complexity from O(N^2 d) to O(NMd) without compromising the flexibility needed for dynamic dependency modeling. This innovation is inspired by the theory of fast high-dimensional Gaussian filtering. Experimental evaluations on benchmark datasets like PEMS-BAY and METR-LA show that EMAGN achieves accuracy comparable to full-attention GMAN, with only a 2.7-3.2% MAE difference, while dramatically improving efficiency. It reduces training time by 32%, inference time by 38%, and GPU memory usage by 58%. Crucially, EMAGN allows for configurations (e.g., 16 attention heads) that would cause full-attention GMAN to run out of memory on standard GPUs, demonstrating a significant expansion of feasible model scales.