Conformal Prediction for Ordinal Classification Using RPS

Ordinal classification (OC) is critical in high-stakes fields like medicine and finance, where accurately quantifying uncertainty must account for the severity of errors across ordered categories. Conformal prediction (CP) offers a powerful framework for generating prediction sets with distribution-free marginal coverage guarantees. However, the practical effectiveness of CP heavily depends on the choice of its nonconformity function, which measures how "unusual" a new observation is compared to training data. This research proposes a new CP method specifically for ordinal classification, utilizing the Ranked Probability Score (RPS) as its nonconformity function. RPS is a proper scoring rule based on cumulative predictive distributions, naturally reflecting ordinal risk. When applied as a nonconformity measure, RPS inherently produces median-centered, contiguous prediction sets. The method is model-agnostic, supports both assessed and grouped ordered categorical outcomes, and is computationally efficient compared to greedy interval selection procedures. Across various ordinal image and tabular datasets, the RPS-based CP method consistently generated contiguous prediction sets and achieved a favorable balance between prediction set width and the magnitude of ordinal miscoverage, demonstrating its superiority over existing CP methods.