New MA-ProofBench Benchmark Evaluates LLM Theorem Proving in Advanced Math

Researchers have developed MA-ProofBench, the first formal theorem-proving benchmark specifically designed for mathematical analysis. This benchmark addresses a critical gap in existing evaluation tools, which often focus on easier-to-formalize areas like algebra and number theory. MA-ProofBench comprises 200 formalized theorems, spanning six core topics and 27 subcategories, including complex analysis and measure theory. The problems are categorized into two difficulty levels: an undergraduate level (Level I) and a Ph.D. qualifying level (Level II), each containing 100 problems. The creation process involved human-led, LLM-assisted formalization, followed by independent expert review to ensure mathematical fidelity. Evaluations of various general-purpose reasoning models and formal theorem provers on MA-ProofBench showed limited success. Even the highest-performing model, GPT-5.5, achieved only 16% Pass@8 on Level I and a mere 5% on Level II, with most other models scoring near 0% on Level II. Analysis indicated that "Mathlib hallucinations" and incomplete proofs were primary failure modes, underscoring a significant disparity between informal and formal mathematical reasoning capabilities in current LLMs.