Imposter: Self-Supervised Learning for Physical Coherence in Scientific Data

Scientific datasets often describe entities whose features are interconnected and governed by physical laws, a concept referred to as physical coherence. However, many existing self-supervised learning (SSL) objectives do not explicitly account for this fundamental property. This research introduces a novel discriminative pretext task called "imposter" to address this gap. The imposter task works by replacing a subset of an entity's features with real observations taken from a different, unrelated entity. The encoder is then trained to identify these "swapped" or "imposter" features. Since each donated value is individually plausible, the model can only succeed by learning the underlying physical dependencies and relationships between different features within an entity. The proposed objectives were evaluated using global ERA5-Land reanalysis data, which includes 21 environmental variables. The learned representations were then assessed on seven downstream tasks, including climate classification, carbon flux estimation, and streamflow prediction. The study, which represents a systematic comparison of SSL objectives for land-surface modeling, found that the most effective pretext task often depends on the specific downstream task family. Importantly, "imposter" was shown to provide complementary information when combined with other existing SSL objectives, highlighting its value in learning physical coherence for scientific foundation models.