Decodability Predicts LLM Hidden-State Selection Efficacy Over Voting

When large language models (LLMs) generate multiple possible answers to a question, combining these into a single, definitive decision is crucial. Traditionally, majority voting is used, but this method often fails on complex questions where sampled answers share similar errors, potentially leading to incorrect conclusions even with more samples. A new approach, Correctness-Axis SElection (CASE), offers an alternative by selecting candidates based on a correctness signal derived from the model's hidden states. The core innovation of CASE is "decodability," a novel, leakage-free metric that predicts whether hidden-state selection will outperform majority voting. Decodability quantifies how effectively a linear gate trained on the answer-token hidden state can rank correct candidates above incorrect ones. Unlike conventional probes, which can appear accurate due to question-identity leakage, decodability provides a robust prediction criterion. Evaluations across general and medical LLMs showed that CASE significantly improves accuracy over voting, particularly on medium-difficulty (up to 19 points) and hard questions (up to 16.8 points). Decodability's predictive power, with a Pearson correlation of r=0.75, allows practitioners to determine in advance when to trust learned selection over simple voting, making LLM decision-making more reliable.