Final Pretraining Window Critically Shapes LLM Post-Training Behavior.

Developers often assume that two LLM checkpoints performing similarly after supervised fine-tuning (SFT) are interchangeable for subsequent alignment stages, such as preference optimization. However, this research challenges that assumption by demonstrating that the final window of pretraining data—even a small fraction of the total—can profoundly shape how a model responds to further training, a phenomenon not immediately apparent from post-SFT benchmarks. A controlled experiment showed that different data sources in the final 500 million tokens of pretraining (e.g., generic web text, safety text, mathematical text) led to near-identical performance after SFT. Yet, when subjected to the same post-training (direct preference optimization or reinforcement learning), these models diverged significantly. Notably, models exposed to safety text last retained far more refusal of harmful requests, an effect specific to the content and its position at the end of pretraining. This suggests that a model's pretraining history, particularly its most recent data, is a critical factor in its alignment potential and should be reported alongside checkpoint evaluations.