Opti-Q Optimizes Multi-LLM Question Answering Under Constraints

Deploying Large Language Models (LLMs) for question answering, especially when using multiple models, faces challenges due to their varying costs, latencies, and energy consumption, alongside their inherent non-determinism. This research introduces OPTI-Q, a novel optimization framework inspired by database query optimizers, which adopts a "plan-before-execute" approach for orchestrating multiple LLMs. OPTI-Q models LLM invocations as operators within an execution directed acyclic graph (DAG). For each question, it searches for plans that maximize answer quality while adhering to user-defined constraints on financial cost, latency, and energy. The framework leverages PERFDB, a catalog of statistics from benchmarks and execution traces, to estimate the performance and resource costs of individual and combined operations. By performing a Pareto-frontier search, OPTI-Q identifies and selects the most efficient plan based on user preferences. Experiments on MMLU-Pro and SimpleQA show OPTI-Q significantly improves average answer quality by 41-58% over baselines at comparable costs.