CircuitSteer Improves LLM Behavioral Control with Multi-Layer Steering

Controlling the behavior of large language models (LLMs) is a significant challenge for AI alignment, with existing methods often relying on fixed, single-layer interventions. These approaches, such as Contrastive Activation Addition (CAA), apply a uniform intervention across diverse inputs and frequently struggle to maintain consistent behavioral changes across different layers, limiting their overall effectiveness. This research introduces CircuitSteer, a new framework designed to overcome these limitations. CircuitSteer leverages Sparse Autoencoders (SAEs) to pinpoint and manipulate specific, coherent semantic circuits that are distributed across multiple layers within an LLM. By analyzing feature co-activation and the geometric alignment of decoder directions, the method isolates the precise multi-layer subcircuits responsible for a target behavior. CircuitSteer then synthesizes dense steering vectors from these identified sparse features and applies multi-point interventions to guide the model's internal semantic trajectory. Evaluations across various tasks—including toxicity, emotion-intensity, sycophancy, and refusal—and two model families demonstrated that CircuitSteer consistently produced fluency-preserving interventions. In contrast, competing methods either compromised text quality or failed entirely on complex behaviors, highlighting CircuitSteer's superior robustness and effectiveness in achieving targeted behavioral control.