New Method Enhances LLM Behavioral Control with Activation Steering

Activation steering is a critical technique for influencing the behavior of large language models (LLMs). Current methods, often based on simple mean differences, struggle to fully capture the complex, non-linear discriminative patterns within the LLM's feature space. This limitation can hinder precise control over model outputs. A new approach, High-Dimensional Random-projection for Activation Steering (HiDRA), has been developed to overcome these limitations. HiDRA integrates with existing steering methods but operates by adding activations in a projected high-dimensional space. This allows it to identify and leverage more nuanced discriminative structures that linear methods cannot detect. Experiments across various LLM families and benchmarks demonstrate that HiDRA consistently outperforms previous baselines. It achieves more robust behavioral control without introducing significant computational overhead, making it a practical enhancement for fine-tuning LLM responses.