LLMs Benchmarked on Floating-Point Error Classification.

Floating-point errors are a common source of bugs in software, particularly in numerical computations, and can be challenging to detect. This research investigates the capability of Large Language Models (LLMs) to statically identify and classify these errors within C programming code, a task traditionally requiring specialized analysis tools. To facilitate this evaluation, the authors developed InterFLOPBench, a new benchmark comprising 90 C kernels and 1,130 test samples. This benchmark is specifically designed to assess LLMs across six distinct categories of floating-point errors: cancellation, comparison, division by zero, overflow, underflow, and NaN (Not-a-Number). The study compared the performance of 14 different LLMs on this benchmark. The evaluation framework treated floating-point error detection as a multi-label classification problem, using the F1-score as the primary metric. Results indicated that the latest generation of LLMs, including Qwen 3 32b, Gemini 2.5 Flash, Phi 4 Reasoning, DeepSeek R1T2, and gpt-oss models, achieved an impressive overall F1-score greater than 0.88. However, performance varied significantly across error categories, with LLMs performing better on explicit errors like division by zero (average F1-score: 0.8479) compared to more subtle numerical phenomena such as underflow (0.6059) and cancellation (0.6164).