Sample Selection Bias Accelerates AI Model Collapse

The increasing reliance on recursive training with synthetic data, while useful for addressing data scarcity, carries a significant risk: model collapse. This phenomenon occurs when repeated training erodes the diversity of data, leading to homogenized outputs and a loss of distributional tails. Data selection is often proposed as a remedy, but its effectiveness is critically dependent on the quality and completeness of the reference distribution used for verification. This paper reveals a crucial vulnerability: in low-resource verification environments, such as healthcare consortia or proprietary financial institutions where raw data cannot be pooled, each verifier only observes a small, fragmented, and inherently biased slice of the target data manifold. Consequently, the selection process itself becomes biased, preferentially retaining samples that align with the local data distribution while inadvertently discarding globally relevant, diverse data points. The research theoretically proves that this 'siloed selection' accelerates model collapse and induces a power-law decay in data diversity. As an initial mitigation strategy, the authors propose constructing Wasserstein proxy references from multiple silos without requiring the sharing of raw, sensitive data. Empirical results confirm that local-reference selection fails on skewed distributions, whereas collaborative proxy references effectively mitigate diversity degradation, underscoring the need for caution in recursive synthetic-data pipelines when real-data coverage is fragmented.