Verification-First Method Improves Language Model Ensemble Accuracy

When combining multiple language models (LLMs) to generate responses, a key challenge is deciding when a new answer from a diverse ensemble should replace an existing, potentially supported one. This research proposes a novel method called Agreement-Before-Diversity (ABD), which decouples the potential for new candidates from the authority to replace an existing answer. ABD operates as a frozen, label-free decision rule. Under ABD, an initial "anchor" answer is kept if two other trusted samples from the ensemble independently corroborate it, based on a predefined equivalence relation. If this agreement is not met, the anchor is replaced by a synthesized response from the heterogeneous ensemble. The method demonstrates significant accuracy improvements on benchmarks like LiveCodeBench-v6 and GPQA-Diamond, outperforming traditional ensemble approaches by prioritizing verified agreement before introducing diverse alternatives. This approach provides a principled, auditable mechanism for coordinating heterogeneous LLMs.