Steering Vectors Control LLM Tool-Use Decisions.

Large Language Models (LLMs) augmented with external tools often invoke these tools unnecessarily, even when parametric reasoning alone would suffice. This study investigates whether there's a stable internal representation within LLMs that governs tool-use decisions, which could then be manipulated. This is a non-trivial question, as tools exist purely in the context at inference time rather than being directly encoded in the model's weights. The researchers found that "steering vectors," derived from specific "heading-anchor" positions within the LLM, can indeed exert bidirectional causal control over tool-invocation behavior. These vectors were effective in suppressing unnecessary tool use across five open-source models and three different domains, particularly in scenarios where the model's inherent knowledge was sufficient. However, a geometric analysis revealed that this causal effectiveness does not correspond to a simple linear structure. Tool-invocation steps showed a diffuse, bimodal alignment with the suppression vector, rather than the consistent negative alignment one might expect from a linear encoding. Furthermore, different tool types appeared to recruit largely distinct internal signatures with minimal cross-tool feature overlap. These geometric properties are hypothesized to reflect the non-parametric nature of tools, distinguishing tool-use steering vectors from those for parametrically grounded concepts. The precise relationship between this geometric irregularity and the observed causal control remains an open area for further research.