Aurora Model Latent Space Encodes Atmospheric Structure

While large language models (LLMs) and other foundation models are increasingly used to emulate complex systems like atmospheric dynamics, their internal workings often remain opaque. This research delves into the Aurora model, a foundation model for atmospheric dynamics, to understand what it encodes in its latent space. Using techniques like spatially pooled Principal Component Analysis (PCA) and layer-wise relevance propagation (LRP), the study found that Aurora's latent space primarily organizes information based on seasonal cycles. Interestingly, extreme storm events do not form distinct, linearly separable clusters in this space. However, LRP analysis revealed that the model pays attention to features consistent with the three-dimensional vertical structure of significant weather phenomena, such as the Great Storm of 1987. Further perturbation tests demonstrated that masking relevant regions in the input degraded Aurora's forecasts significantly more than random masking, by a factor of 3.31x. These findings suggest that the Aurora model implicitly learns meteorological coherence and the vertical structure of the atmosphere, even without explicit programming or instruction to do so. This sheds light on how such "black box" models develop their understanding of complex physical systems.