Constitutional Value Potentials Read and Steer LLM Priorities

While a "constitution" can instruct a language model on its values, assessing whether the model truly adheres to these values, especially during conflicts, remains challenging. Traditional methods rely on output analysis, which can be unreliable when models must prioritize one value over another. This paper presents a novel approach called Constitutional Value Potentials (CVP) to address this by directly reading internal priority margins from the model's activations. CVP involves learning a scalar potential for each value from the hidden state of the language model. This potential represents an internal pressure to preserve that value, and it is supervised not by the prompt itself, but by an independent judge's assessment of which value the model's response actually upheld. The signed difference between two potentials then indicates a priority margin. A constitutional clause can be framed as the claim that a specific margin remains positive, and a single monitor score can flag when this condition is violated. The CVP monitor demonstrates high predictive accuracy, achieving an AUROC of up to 0.95 for conflict violations, outperforming strong hidden-state probes. This signal emerges early in the generation process, from the prompt tail and the first response token, allowing for early detection of potential violations or adversarial manipulations. Furthermore, the identified value directions in activation space can be used for intervention tests, showing that moving along these directions can shift the model's judged trade-offs in the intended direction, suggesting a powerful mechanism for steering model behavior.