SDO Optimizes Data Organization for Efficient LLM Post-Training

Post-training of large language models (LLMs) is a resource-intensive process, with existing efficiency improvements often focusing on sample selection or training schedules. However, the way training data is organized into mini-batches is typically a static preprocessing step, failing to adapt to the dynamic optimization needs of samples as training progresses. This can lead to redundant updates for some samples while others remain under-optimized, hindering overall convergence and efficiency. To address this, a new framework called SDO (Structure-Aware Data Organization) has been introduced. SDO is a plug-and-play solution that dynamically organizes data based on its representation-space structure and an exposure-driven feedback mechanism. It operates epoch by epoch, using frozen external embeddings to avoid warm-up overhead. Within each epoch, locality-aware batching forms coherent mini-batches via KNN neighborhood traversal. Across epochs, SDO employs exposure-balanced scheduling, recording per-sample participation and reducing the sampling probability of over-exposed samples. This ensures long-term coverage and more balanced optimization. Experiments across SFT, DPO, and GRPO tasks show that SDO accelerates convergence, particularly in the early-to-mid phases, by producing more coherent gradients and achieving more balanced accuracy across different question types without permanently excluding any training samples.