Sharpness-Guided Sampling Improves Long-Tailed Learning Generalization

Long-tailed learning presents a dual challenge: head classes dominate training data, leading to biased models, and under-represented tail classes often converge to sharp, less generalizable regions of the loss landscape. Existing methods typically address these issues separately; re-sampling tackles class imbalance without considering geometry, while sharpness-aware minimization (SAM) modifies losses or perturbations only after biased mini-batches are already drawn. This paper introduces Sharpness-Guided Equilibrium Sampling (SGS), a new approach that treats the sampling distribution itself as an active control variable for optimization geometry. SGS dynamically adjusts subsequent mini-batches by increasing the sampling probability of less frequent classes while simultaneously suppressing classes that exhibit large SAM-induced loss changes. This is achieved using only cumulative class counts and Exponential Moving Average (EMA) sharpness estimates derived from standard SAM updates, avoiding the need for class-wise perturbations or extra backward passes. The researchers characterize this sampling process through a continuous-time stochastic differential equation and a sampling-dependent PAC-Bayes analysis, explaining how this frequency-sharpness feedback mechanism guides training towards a more balanced flatness profile. SGS-SAM demonstrated significant improvements, boosting tail accuracy by 10.85 points and overall accuracy by 3.56 points on CIFAR-100 LT, and similar gains on ImageNet-LT, all with only a marginal increase in training time. This work highlights a new "sampling-side" route to controlling loss-landscape geometry, suggesting future long-tailed methods can jointly regulate data exposure and optimization geometry.