AI Leaderboards Primarily Track Time, Not Distinct Economic Capabilities

Frontier AI leaderboards, which rank models based on performance across various professional tasks from software engineering to banking, heavily influence purchasing decisions, regulatory scrutiny, and expectations about the future of work. A new study investigates the construct validity of these benchmarks, questioning whether they measure distinct economic capabilities or merely reflect a single, overarching axis of improvement over time. Analyzing 421 model configurations across twelve benchmarks, including four economic ones, researchers found that a single factor accounts for 74.5% of the common variance and strongly correlates with model release date. This suggests that the leading axis of capability is substantially a time trend, with compute adding little once the date is controlled for. While a multi-factor representation did offer incremental predictive information for held-out economic scores, the evidence does not support treating economic benchmarks as measuring a truly distinct latent capability. Therefore, while leaderboards remain a sound guide to overall progress, much of the performance difference between models released months apart is attributable to the calendar, implying that small gaps between contemporaneous models should be date-adjusted before being interpreted as significant capability differences.