LinearKV Optimizes Caching for Hybrid LLMs

Large Language Model (LLM) serving performance is increasingly enhanced by position-independent caching (PIC), which reuses token chunks. However, existing PIC methods are designed for full-attention models and struggle with "hybrid LLMs" that incorporate linear recurrences, as these models expose only a fixed-size state rather than a token-indexed KV cache. This paper introduces LinearKV, a novel training-free framework that brings PIC benefits to hybrid models. The core innovation of LinearKV is "decoupled initialization." For linear layers, it maps multiple matched local states to a single initial state, while full-attention layers continue to concatenate their KV as before. Crucially, the research finds that a *single cached state* is sufficient as the linear layer's initializer. This simpler approach is not only more efficient but also, on some architectures like Mamba-2, significantly outperforms the algebraically principled but more complex method of composing all cached states into an exact full-prefix state. LinearKV significantly cuts the time-to-first-token, achieving 0.46 times the full prefill time, compared to a 5-17% overhead for exact composition. These performance gains and quality preservation were demonstrated across various hybrid models and PIC selectors on benchmarks like LongBench QA and RULER, highlighting its practical value for accelerating hybrid LLM serving.