Causal Direction Benchmarks Re-evaluated with New Parameter-Free Baseline

The field of causal direction inference often compares methods using headline accuracies from the Tuebingen dataset, but these comparisons are flawed. Each method is typically evaluated under its authors' specific protocols, which vary in pair subsets, weightings, model selection, and decision rates. This inconsistency makes direct comparison unreliable and can inflate reported performance. Researchers conducted a "same-hands" re-evaluation, running all methods on an identical set of 102 pairs with a strict rule: no tuning and a forced decision for every pair. They also introduced a minimal, parameter-free baseline using sorted-conditional compression, which processes quantized, sorted, and differenced data with a standard compressor. Under this standardized "common ruler," the ranking of methods significantly differs from published literature. The simple compression baseline achieved 74.7% weighted accuracy, performing comparably to more complex, tuned methods. The study highlights mechanisms that inflate published figures, such as test-set model selection and significance-gated abstention, and provides a more accurate, consistent benchmark for future causal inference research.