TKG Forecasting Models Struggle with Distribution Shifts

Researchers have conducted a systematic evaluation of Temporal Knowledge Graph (TKG) forecasting models, specifically examining their robustness when confronted with distribution shifts in the underlying data. Recognizing that empirical benchmarks offer limited insight into model resilience, the study utilized a synthetic TKG generator. This generator allowed for controlled manipulation of three key temporal and structural properties: recurrence, homophily, and periodicity, enabling the creation of both stationary and shifting data regimes. The experiments, involving seven different forecasting architectures, revealed that while models generally perform well under stationary conditions, particularly in recovering recurrence-based and periodic regularities, their adaptivity is severely tested by structural breaks. Shifts in the latent entity-community structure emerged as the most significant challenge, indicating a fundamental limitation in how current TKG models handle evolving relational systems. The findings underscore that robustness in TKG forecasting is highly dependent on the specific type of signal and shift, providing crucial insights into the capabilities and weaknesses of existing models.