PIT-SUN Improves Recommender System Regression for Heavy-Tailed Data

Estimating original-space conditional expectations, such as dwell time or customer lifetime value, is fundamental for value-driven recommender systems. However, standard Mean Squared Error (MSE) loss functions often struggle with heavy-tailed, zero-inflated, or multimodal target distributions, leading to unstable gradients, mean collapse, and tail shrinkage in predictions. While target transformations can alleviate these issues, direct inversion of non-linear transforms loses expectation consistency. This research introduces PIT-SUN (Probability-Integral-Transformed Unbiased recovery), a deployable empirical marginal recovery framework designed to restore expectation consistency while handling complex target distributions. PIT-SUN defines a bounded normal-score coordinate using an empirical marginal table, which also provides its inverse-quantile lookup, a variance-controlled recovery base, and drift monitoring. Instead of directly inverting transformed predictions, PIT-SUN applies multiplicative SUN recovery to estimate the original-space expectation. Experiments across synthetic distributions, public benchmarks, large-scale industrial datasets, and online deployments demonstrate robust improvements. The framework enhances point accuracy, calibration, and ranking quality with minimal deployment overhead, making it a practical solution for real-world recommender systems.