ASI-Bench: New Benchmark for AI Scientific Exploration and Autonomy

The concept of Artificial Superintelligence (ASI) implies AI systems that can not only master existing knowledge but also explore the unknown, create new knowledge, and validate novel ideas. Current AI benchmarks primarily assess an AI's ability to apply learned information or complete tasks with significant human guidance. To address this gap, a new benchmark called ASI-Bench has been developed. ASI-Bench is the first benchmark to jointly evaluate AI's capacity for innovative exploration and autonomous scientific execution across a wide range of research domains. It features 60 project-level research tasks spanning 11 scientific fields. A key aspect of ASI-Bench is its progressive reduction of human methodological guidance, challenging AI systems to independently select methods, conduct research, and produce verifiable results. Developed by over 40 experts and requiring more than 31,000 human hours, the benchmark tasks undergo rigorous review, auditing, and validation. Initial evaluations of 18 state-of-the-art agent-model configurations showed a significant drop in performance as human guidance decreased. This stark decline indicates that contemporary AI systems are still far from autonomously conducting end-to-end scientific research. ASI-Bench is now open to the research community for contributions and challenges.