CausalGate Prunes Transformers for Efficient Inference.

Current adaptive inference methods for large language models (LLMs) often rely on observational heuristics, such as hidden-state similarity or activation magnitudes, to identify and drop redundant modules. However, these correlation-based metrics frequently fail to capture the subtle, non-linear structural computations that are crucial for maintaining semantic accuracy. Researchers have introduced CausalGate to address this limitation. CausalGate is an intervention-guided framework designed for compute-efficient transformer inference. During a calibration phase, the framework systematically isolates individual Attention and MLP sub-layers. It then zeros out their respective outputs and precisely quantifies the semantic damage caused by this intervention, using the Kullback-Leibler divergence of the final logit distribution as a metric. This process establishes a clear structural importance hierarchy. To eliminate runtime routing overhead, this importance hierarchy is subsequently distilled into a global set of static, lightweight scalar gates. This distillation is achieved using an Exponential Moving Average smoothing objective combined with a differentiable pairwise ranking loss. Evaluated on models like TinyLlama-1.1B, Qwen2.5-3B, and Llama-3.1-8B across various benchmarks, CausalGate consistently outperforms prominent dynamic routing and layer-skipping baselines. It translates theoretical compute savings into tangible hardware latency reductions without incurring any additional operational overhead.