New Framework Improves LLM Clarification by Measuring Uncertainty

Large language models often struggle with ambiguous or incomplete user queries in open-domain interactions, frequently leading to inaccurate or unhelpful responses. The traditional solution of asking clarifying questions is effective but typically relies on expensive manual annotations or preference alignment to determine when and what to clarify. This reliance limits scalability and generalization. A new framework, CLAIM (Clarification Learning with Uncertainty Measurement), addresses these challenges by eliminating the need for explicit human preference annotations. CLAIM quantifies query uncertainty by measuring the entropy of answer disagreements among multiple models. This uncertainty signal is then used to create high-quality synthetic data, which trains a unified clarification decision model. The training process combines supervised fine-tuning with group-relative policy optimization, formulating clarification as a structured decision generation problem. Experimental results demonstrate that CLAIM can learn stable and generalizable clarification strategies, offering a robust and cost-effective solution for LLMs to proactively understand user intent in real-world, open-domain conversations.