New Attack Method Probes LLM Safety Representations.

Behavioral alignment in large language models (LLMs) often conceals fragile internal safety mechanisms. Previous work suggests that an LLM's refusal to generate harmful content is mediated by specific low-dimensional directions within its activation space. This raises critical questions about the structure, localization, and accessibility of these safety representations to optimization. This paper investigates adversarial suffix attacks as a means to probe these representational alignments. It introduces Activation-Guided GCG, an approach that replaces traditional output-based objectives with losses directly targeting the model's internal "refusal direction." The findings indicate that suppressing refusal across all layers and positions is more effective than targeting a single layer, suggesting that safety representations are distributed throughout the model's forward pass. Additionally, Soft-GCG, a continuous relaxation of discrete suffix optimization, is introduced, achieving a 33x speedup and improved attack success rates. Evaluation across model scales shows smaller models remain vulnerable, while larger, better safety-trained models exhibit greater resistance, consistent with increased robustness. These insights offer guidance for designing more resilient and representation-aware alignment strategies.