New RL Method Improves Probabilistic Forecasting Calibration.

A new research paper introduces an innovative method for training calibrated probabilistic forecasters using reinforcement learning (RL), specifically addressing the challenges in aleatoric forecasting where the outcome is stochastic. Traditional RL approaches often degrade calibration when using verifiable rewards like the Brier score, especially when dealing with noisy, single-outcome labels. The study highlights that directly rewarding per-play outcomes in scenarios like NFL win probability leads to poor calibration due to label noise and corruption of the policy gradient. To overcome these limitations, the researchers propose a verifiable, label-free reward mechanism based on a state-conditioned empirical win rate, estimated from past outcomes. This approach effectively removes label noise. Furthermore, they prevent gradient corruption by either direct prediction or applying a gradient mask, ensuring the model's reasoning chain remains intact. When trained solely with this new reward, a 7B model achieved calibration levels comparable to the betting market for NFL in-game win probability, outperforming zero-shot frontier models. The findings suggest that masking the gradient is crucial for preserving the model's reasoning, which is often compromised in standard chain-of-thought training.