XMerge Compresses LLMs by Merging Layers, Retaining Quality

Reducing the number of layers in large language models (LLMs) is a common strategy for depth compression, which helps maintain a standard serving architecture and reduces inference costs. However, existing methods often lead to significant and unpredictable quality loss. This paper introduces XMerge, a novel post-training approach designed to overcome these limitations. XMerge operates with two main components. First, it employs a "cross-axis selection" mechanism to identify a block of layers with minimal relative-magnitude and angular hidden-state change, indicating it can be removed with less impact. Second, it uses "local boundary reconstruction" to re-fit the adjacent surviving block, ensuring its output closely matches the original two-block output. Crucially, XMerge requires no task labels, end-to-end fine-tuning, architectural changes, or additional inference-time parameters. Experiments across seven Llama and Qwen models (0.5B-8B) demonstrate that XMerge consistently outperforms five published baselines, especially at aggressive layer reduction levels. It ranks first on multiple aggregate benchmarks and avoids the catastrophic perplexity increases seen in competing methods, making it a robust solution for efficient LLM deployment.