BenchPress Predicts LLM Performance with Fewer Evaluations

Evaluating large language models (LLMs) typically involves running them across dozens, sometimes hundreds, of benchmarks. This process is resource-intensive, consuming significant computational power and time, both during development and for final model releases. A new study analyzed a public score matrix of 84 frontier models across 133 benchmarks and discovered a surprising underlying structure: the performance of a model across all these evaluations can be largely explained by just two latent factors. This means that a model's comprehensive scorecard is effectively determined by only two numbers. Building on this insight, the researchers developed BenchPress, a logit-space rank-2 matrix completion method. BenchPress can recover held-out scores to within 4.6 points and provides a confidence layer for its predictions. It identified a minimal subset of five benchmarks (e.g., GPQA-D, HLE, Codeforces, MMLU-Pro, ARC-AGI-1) that can predict a model's full public scorecard within 3.93 points, and an even cheaper set for tighter inference budgets. This tool promises to drastically reduce the computational burden of LLM evaluation.