DRIFT Refines LLM Training Data for Enhanced Capability

Optimizing the distribution of training data is crucial for enhancing the capabilities of Large Language Models (LLMs through Supervised Fine-Tuning (SFT). While existing data curation methods effectively accelerate training under budget constraints, they often fall short in elevating the model's ultimate performance ceiling. The core challenge lies in identifying and refining data instances that are most effective at improving the final model, rather than just preserving current performance. This research explores instance-level data attribution using Influence Functions (IF) to tackle this problem. The authors identify two key limitations in standard IF formulations: a "proximity gap" arising from off-policy validation targets and a significant bias towards gradient norm. To overcome these, they propose DRIFT (Data Refinement via On-Policy Influence Functions for Supervised Fine-Tuning). DRIFT uses the model's own on-policy rollouts as validation targets, which empirically reduces the parameter proximity gap and better aligns with IF's local neighborhood assumption. It also applies signed weighting based on trajectory correctness and debiases influence scores, allowing a small set of validation queries to reliably attribute the entire dataset. Experiments on 7B-parameter instruction and reasoning models demonstrate that DRIFT consistently improves performance, surpassing existing data curation baselines.