MiMo-V2.5 LLM Inference Optimized for Hybrid SWA Efficiency

Researchers have developed a full-pipeline inference optimization strategy specifically for the MiMo-V2.5 model family, which combines advanced architectural elements like Hybrid Sliding Window Attention (SWA), sparse Mixture-of-Experts (MoE), and multimodal encoders. While Hybrid SWA theoretically offers substantial reductions in attention computation and KVCache storage, realizing these benefits in a production environment requires significant engineering effort. The optimization effort systematically targets the KVCache system, implementing layerwise prefetch, SWA-aware prefix cache trees, and specialized placement strategies. This ensures strict O(W) SWA storage complexity and high cache hit rates, maximizing the efficiency of the sliding window mechanism. Further enhancements include the development of GCache, a high-performance distributed cache infrastructure utilizing RDMA-optimized networking, alongside a KVCache-affinity router. This router is designed to minimize computation while maintaining effective load balancing across the system. The optimization also extends to multimodal inputs, incorporating GPU image preprocessing, parallel video decoding, and shared multimodal caching. Collectively, these advancements represent the first large-scale LLM serving system in production that efficiently supports the complex composite architecture of Hybrid SWA, MoE, and multimodal inputs. This work demonstrates how to push the boundaries of inference efficiency for sophisticated, multi-component LLMs.