LLMs Prioritize Context or Memory Through Activation Directions

A recent study delves into the intriguing question of how large language models (LLMs) resolve conflicts between information provided in the input context and knowledge stored within their parameters (memory). The research demonstrates that by analyzing activation directions, it's possible to both decode and influence which source of information an LLM prioritizes. Specifically, "authority directions" were estimated from prompts where context and parametric knowledge aligned, and then these directions were manipulated in counterfactual experiments. The findings indicate that interchanging naturally occurring coordinates along these authority directions between matched prompts can reproduce a substantial portion (30-68%) of the model's decision to favor either context or memory across models like Qwen, Llama, and OLMo. However, the study also highlights limitations regarding cross-task reusability. Authority directions learned on one task showed only about 9% transferability to another task, compared to 57% when using a direction learned specifically for that task. This suggests that while authority representations exist and can causally influence source choice, the underlying computations may be highly task-dependent rather than universally reusable.