Fine-Tuning Strategy Boosts Hallucination Detection in LLMs

Hallucination detection in large language models (LLMs) is typically performed at the inference stage without altering the model itself. However, many erroneous outputs remain undetected because models often produce nearly identical incorrect answers across multiple runs, leading to low semantic entropy and thus, missed hallucinations. This research explores fine-tuning strategies specifically designed to improve the detectability of hallucinations. The proposed "diversity-oriented fine-tuning" aims to encourage more varied generations from the model. Two specific strategies are introduced: one based on Supervised Fine-Tuning (SFT) and another on Direct Preference Optimization (DPO). Extensive experiments demonstrate that these fine-tuning methods make models less likely to produce low semantic entropy responses for hallucinated answers. This significantly improves the effectiveness of uncertainty-based hallucination detection, yielding results that are comparable to or better than state-of-the-art methods. The code for this approach will be publicly released, offering a new direction for enhancing LLM reliability.