Diverse LLM Crowds Outperform Larger Undiverse Groups in Prediction.

This paper introduces a novel framework for building effective "wisdom-of-the-crowd" systems using large language models (LLMs) for future prediction tasks. The core idea is that simply using more LLMs doesn't guarantee better predictions if those models exhibit redundant behaviors. Instead, the proposed method focuses on creating diverse crowds by analyzing the reasoning traces of different LLMs on independent development tasks. The framework clusters models based on their behavioral similarity and then selects representative models for collective prediction. Experiments involving 25 LLMs across various benchmarks showed that the composition of the crowd is more critical than its sheer size. A small, three-model crowd, selected using K-means++ behavioral clustering, significantly outperformed a conventional voting system involving all 25 models. This approach not only yielded superior prediction accuracy but also dramatically reduced computational resources, cutting model calls by 88% and inference costs by approximately 80%. The findings suggest that focusing on representative behavioral diversity is key to constructing highly effective and efficient LLM crowds for predictive tasks.