DecodeShare Reveals Shared LLM Decode-Time Subspace

Researchers have introduced DecodeShare, a novel protocol designed to identify a low-dimensional subspace within the hidden states of large language models (LLMs) that is consistently shared across various tasks during the decode-time inference phase. While LLMs use a single set of parameters for many tasks, the specific task-general structure utilized during decoding, distinct from prefill, has been unclear. DecodeShare not only identifies this shared subspace but also establishes its causal importance: experimentally disturbing this subspace during decoding leads to a significantly greater degradation in decision performance compared to disturbing prefill-derived or random subspaces. Furthermore, the study reveals that this decode-shared subspace has practical implications for activation steering, as common steering directions can overlap with this task-general channel. Evaluating steering vectors at decode-time provides more reliable signals for deployment than traditional prefill-based proxies, indicating that this compact shared subspace acts as a high-leverage causal channel during decoding.