HyperDFlash Boosts LLM Decoding Speed with MHC-Aligned Speculative Decoding

This research introduces HyperDFlash, a novel speculative decoding framework specifically engineered for the DeepSeek-V4 model's unique multi-hyper-connection (MHC) architecture. While DeepSeek-V4's native multi-token prediction (MTP) module shows strong initial performance, its accuracy quickly declines due to error accumulation in later tokens, hindering acceptance rates. Existing speculative decoding methods like DFlash are not directly compatible with MHC due to its multi-path residual stream, causing feature misalignment. HyperDFlash addresses these challenges through two key optimizations. It uses pre-collapse residual states as the primary conditioning signal, preserving MHC's structural information and aligning the drafter with the target model's prediction pathway. Additionally, it replaces a heavy linear compressor with a lightweight gated residual reducer, which inherits parameters from the model's built-in hyper-connection head, ensuring architectural alignment with significantly fewer parameters. The framework also incorporates a targeted KL distillation loss to regularize predictions and improve draft quality during early training. Extensive experiments across various benchmarks, including math reasoning, code synthesis, and conversational tasks, demonstrate HyperDFlash's superior performance. It consistently surpasses both DeepSeek-V4's native MTP baseline and adapted DFlash versions, achieving substantial improvements in average accepted draft length and overall decoding speedup. This validates the effectiveness of its MHC alignment, gated reduction, and targeted distillation techniques for high-performance speculative decoding.