Trajectory Data Quality Impacts Code Agent Fine-Tuning

Fine-tuning open-weight Large Language Models (LLMs) with expert agent trajectories is a popular method for developing capable code agents, especially without relying on proprietary models. However, the precise impact of trajectory data quality and quantity on model performance has remained largely unexplored. This research provides a systematic empirical study on how filtering trajectory data affects LoRA fine-tuning for code agents. The study introduces a two-axis quality scoring framework, evaluating trajectories based on "Efficiency" and "Style." Through 16 controlled experiments, it investigates the interplay of strategy, scale, and various ablations. Given the low resolve rates of 7B-scale models on SWE-bench, cross-entropy (CE) loss on held-out trajectories is used as the primary metric, validated by its strong correlation with ROUGE-L. Key findings reveal a scale-dependent trade-off between quality and quantity. At smaller dataset scales, doubling the quantity yields significant performance gains, while at larger scales, the quality gap (e.g., between top-quality and random trajectories) becomes more pronounced. The error-retry rate is identified as a particularly influential sub-dimension of quality. These results underscore that trajectory-level quality scoring is a viable, though scale-sensitive, lever for fine-tuning code agents.