Stochastic Sampling Lacks Deep LLM Diversity

New research suggests that the common practice of using stochastic sampling (e.g., varying temperature) in large language models to generate diverse outputs is epistemically shallow. While this method can provide a measure of uncertainty for individual questions through self-consistency, it fails to reveal deeper, structural knowledge gaps across different, related questions. The study compared a single LLM run multiple times with temperature variation against an ensemble of different LLMs, each run once. The findings indicate that a single model, even with varied sampling, exhibits at most one dimension of signal above noise, suggesting limited internal diversity. In contrast, a diverse ensemble of models showed multiple dimensions of signal, effectively surfacing a broader range of what the models collectively do not know. This implies that true epistemic diversity and the ability to identify complex knowledge gaps require multiple distinct models rather than just repeated sampling from one.