Multilingual Verifier Bias Impacts RLVR in LLM Mathematical Reasoning

New research uncovers a critical issue in training large language models (LLMs) for mathematical reasoning using Reinforcement Learning with Verifiable Rewards (RLVR) in multilingual contexts. The study demonstrates that the common assumption of a language-neutral reward function, typically an exact-match verifier, fails when applied across different languages. This failure manifests as significant language-dependent false-negative reward noise, primarily due to variations in answer formats and scripts. The researchers introduced a protocol to audit multilingual RLVR rewards, including a verifier-robustness suite and rollout diagnosis. Their findings show that exact-match verifiers reject trusted-correct answers at sharply different rates across languages (e.g., Japanese, English, Chinese) for models like Qwen3 and Llama-3.1. For Qwen3-8B, the false-negative rate for Japanese answers was substantially higher than for English or Chinese. This bias is localized to the final-answer interface and points to a "cross-lingual selection bottleneck," where effective repairs require genuine cross-lingual support, underscoring the need for language-specific auditing and optimization of RLVR rewards.