Uncertainty-Gated Selection Boosts Block-Sparse Attention

Block-sparse attention is a technique used to scale long-context language models by reducing the quadratic complexity of softmax to a per-query top-k selection over key blocks. However, this approach can be myopic; if the k-th and (k+1)-th blocks have very similar scores, the system commits to the top-k without further consideration, potentially dropping crucial information. Researchers have introduced a novel value-of-information router to address this limitation. This router measures the decisiveness of the top-k cut for each query. For queries where the gap between the k-th and (k+1)-th block scores is smallest (indicating high uncertainty in the selection), the router dynamically doubles the kept set of blocks. This mechanism is backbone-agnostic and can be stacked with existing block-scoring methods, such as Quest. Evaluations on LongBench-v2 medium demonstrated that the router-on-Quest approach achieved a paired recall of 0.75, a significant improvement of +28 percentage points over the SSA-style baseline's 0.47. It also performed within 2 percentage points of dense attention on RULER NIAH multikey at the same context. The performance lift was consistent across four models from three different architectures (Qwen2.5, Mistral-Nemo, Qwen3.6). At 128K context, the router preserved 0.81 and 0.89 of dense accuracy on Qwen2.5-7B-1M and Qwen3.6 respectively (compared to 0.09 for SSA-style top-k), while the fused selection-plus-kernel pipeline maintained efficiency, running at 0.62x and 0.80x dense wall time.