LLM Safety Mechanisms Vulnerable to Low-Resource Language Attacks

Large language models (LLMs) are primarily trained for safety in English, creating a significant vulnerability when interacting with low-resource or mixed-language inputs. This training gap means models can confidently generate harmful responses for prompts outside their safety training distribution. Researchers developed STEER (Safety Targeted Embedding Exploit via Refinement), a gradient-guided attack that identifies words crucial to a model's refusal behavior and iteratively translates them into low-resource languages. This process effectively bypasses safety filters while retaining the harmful intent of the original prompt. STEER achieved high attack success rates, up to 93.0% on JailbreakBench and 96.7% on AdvBench, across six open-source 8B-parameter models, outperforming other methods. The prompts also transferred to GPT-4o-mini with a 35.5% success rate, indicating a broader architectural weakness. These findings highlight that current English-centric safety mechanisms are insufficient for multilingual contexts, emphasizing the need for broader language coverage in alignment and explicit detection of out-of-distribution inputs.