Lightweight Multimodal Model Predicts LLM Document Reasoning Performance.

Uniformly allocating reasoning budgets to Large Language Models (LLMs) for document tasks is often inefficient and costly, especially given that visual layouts significantly influence task complexity. To address this, a new multimodal benchmark called BudgetDoc has been introduced. BudgetDoc provides explicit supervision for understanding the trade-offs between model budget and performance across three distinct document tasks. Leveraging BudgetDoc, researchers developed DRB (Document-Reasoning Balancer), a relatively lightweight multimodal estimator with approximately 1 billion parameters. DRB, composed of SigLIP-2 and Qwen3-0.6B, is designed to predict the ordinal performance of LLMs across various budget levels. It achieved a weighted F1 score of 0.753 in its predictions. When DRB was used to dynamically allocate reasoning budgets across five frontier LLMs and three datasets, it either matched or improved F1 scores in 9 out of 15 configurations compared to always using the maximum budget. Crucially, this dynamic allocation drastically reduced computational costs. Preliminary evaluations also suggest DRB's potential to generalize to cross-model selection, offering a promising path towards more compute-optimal document inference.