New OSW-FT Method Improves LLM Fine-Tuning Efficiency

Researchers have introduced Online Self-Weighted Fine-Tuning (OSW-FT), a straightforward yet effective method designed to improve the efficiency and performance of supervised fine-tuning (SFT) for large language models (LLMs). Traditional SFT methods apply uniform loss weights to all expert demonstrations, regardless of the model's evolving competence during training. OSW-FT addresses this limitation by augmenting SFT with an online, trajectory-level weighting mechanism. For each training query, the method estimates the model's current success rate using a small number of inference-only rollouts, typically just two, and then rescales the standard SFT loss accordingly. This adaptive weighting ensures that the magnitude of updates adjusts dynamically to the model's performance, while the optimization direction remains anchored to the expert trajectory. The approach is particularly beneficial for binary-verifiable reasoning tasks and has been shown to consistently outperform standard SFT on small-to-medium scale models, such as the Qwen3 series (0.6B to 4B), across various challenging benchmarks. OSW-FT offers a favorable compute-performance trade-off, making it a practical solution for fine-tuning LLMs with limited computational resources.