Hierarchical Data Selection Boosts LLM Fine-tuning Efficiency

Supervised fine-tuning of large language models (LLMs) often involves vast datasets, making efficient data selection critical for managing costs and enhancing model efficacy. Current methods for choosing data subsets typically assess diversity in the original embedding space, which can conflate various semantic signals and noise. This new approach, called MASS, tackles this by reframing data selection as a hierarchical coverage challenge. MASS operates in two stages: first, it learns low-dimensional principal manifold coordinates using a dense autoencoder to group data semantically at a coarse level. Second, within these groups, it applies a quality-aware sparse feature coverage using a TopK sparse autoencoder. This allows for a more refined selection process. Evaluations on datasets like Vision Flan and LLaVA-CoT demonstrate that MASS consistently outperforms existing data selection techniques across different budget constraints. In several scenarios, it achieves or even surpasses the performance of training with the full dataset, but with only a fraction of the data.