Minima-KV Compresses LLM Cache for Longer Contexts

Researchers have developed Minima-KV, a novel system designed to compress the Key-Value (KV) cache in large language models (LLMs) to support longer context windows more efficiently. The KV cache is a significant bottleneck in terms of memory capacity and bandwidth for long-context LLM serving. Minima-KV employs a mixed-format paged attention hierarchy where recent and protected "Anchor" pages remain in high-precision FP8, while older, less critical pages are compressed into a packed TQ3 format. This approach allows all live-request pages to remain addressable, with format-specific kernels computing partial attention states that are then combined through a globally normalized online-softmax merge. This enables direct heterogeneous decoding without needing a large, dense shadow cache. Benchmarks on a Qwen3.6-27B model demonstrated 3.50x compression relative to BF16 and 1.75x relative to FP8, with minimal impact on output quality across various long-context tasks. The system achieved 3.625x active-KV compression and 0.9821x throughput for very long requests, proving a practical path for compressing long-context state without evicting live KV pages.