New Method Prevents LLM Factual Access Failures Post-SFT

This research delves into a specific problem encountered after supervised fine-tuning (SFT) of Large Language Models: "factual access failure." This phenomenon occurs when an LLM, despite retaining factual knowledge (demonstrated by its ability to recognize or rank correct answers in constrained settings), fails to produce that information in open-ended generation tasks. The study attributes these failures not just to catastrophic forgetting, but also to expression-level issues like verbosity, formatting mismatches, or exact-match artifacts. To combat this, the researchers introduce Recall-Anchored Distillation (RAD). This novel self-distillation objective aligns the adapted model with the original base model's soft continuation distribution on unlabeled out-of-distribution (OOD) text. Crucially, RAD does not require gold OOD answers, external judges, or labeled factual data. Experiments across three model backbones fine-tuned on MedMCQA show that RAD consistently recovers a significant portion of lost OOD factual recall while maintaining target-domain adaptation.