SCX Router Enables Zero-Shot LLM Selection for Optimal Performance

The proliferation of large language models (LLMs) and their diverse applications creates a significant opportunity to optimize by selecting the most suitable model for each specific task, balancing speed, cost, and quality. However, the wide variations in performance, pricing, latency, context handling, tool integration, domain expertise, and reasoning capabilities among different inference endpoints make manual selection and maintenance challenging. This paper introduces the SCX Router, a compact, GLiClass-based routing system that assigns a suitability score to each potential inference model without requiring autoregressive generation. The SCX Router, built on a 0.6B-parameter Qwen3 decoder combined with a shallow bidirectional scorer, features a decoder-KV execution path that maintains a text-only key-value cache across sessions, processing only new dialogue turns and evaluating candidate-label tokens without adding them to the persistent cache. This checkpoint also predicts task type, difficulty, reasoning mode, and expected output length, supporting custom zero-shot labels. To facilitate task generation, the researchers developed a comprehensive task ontology comprising 23 families, 115 task types, 345 routable subtypes, 1,173 synthetic examples, and 30 orthogonal domains. Using this structure, they generated 150,000 verifier-scored tasks and 15,000 open-ended tasks, training the Qwen3 decoder while explicitly separating learned request prediction from per-task policies for attributes like eligibility, cost, cache reuse, safety, and sovereignty. Across six LiveBench subsets, the router consistently outperformed the average candidate, achieving an aggregate top-1 score of 0.707 on a 1,000-task subset, surpassing the strongest fixed model's 0.696 score, with benchmark-dependent gains.