New Metric Validates Causal Explanations in Complex Systems

The scientific community lacks a consensus on how to quantitatively measure the validity of high-level causal explanations for complex systems. This research addresses this gap by creating a benchmark of ten diverse complex systems, encompassing both discrete and continuous states, and static and dynamic behaviors. Each system is equipped with known ground-truth causal explanations and deliberately invalid contrastive conditions. Within a unified causal abstraction framework, the study systematically evaluates more than thirty candidate metrics from various families, including observational, functional, information-theoretic, and causal approaches. The findings indicate that only causal metrics, specifically those incorporating faithfulness testing over unmapped variables, consistently distinguish valid from invalid abstractions. Building on these insights, the authors introduce the Causal Abstraction Error (CAE), a continuous validity metric that includes an explicit faithfulness test. CAE successfully passes all discrimination tests across every system in the benchmark and can converge effectively with as few as 30 sampled interventions. This metric is offered as a general-purpose tool for discovering and validating high-level explanations in complex domains.