DQC Improves Multi-Modal Regression by Resolving Mean-Collapse

Multi-modal regression, where a single input can correspond to multiple valid outputs, often suffers from a "mean-collapse" issue. Traditional regressors, optimized with squared loss, tend to predict the conditional mean, which typically doesn't align with any of the actual output modes. This research identifies pairwise contradictions (similar inputs, distant outputs) as the root cause and proposes a solution called Difference-Quotient Clustering (DQC). DQC works by partitioning the data to minimize the discrepancy between outputs and inputs within each cluster. Each data sample is assigned to the cluster that minimizes its maximum contradiction ratio. This process generates coarse conditional labels, which are then used to train a logits generator and a conditional network. On synthetic datasets, DQC achieved significantly better minimum squared error compared to random labels or mean-collapse, demonstrating its ability to resolve the multi-modal challenge.