Fool's Gold Deceives Safety-Removal Attacks on Open-Weight Models

Open-weight language models face a significant vulnerability: their safety alignment can be easily removed, often within minutes, by attackers. Existing defenses have largely failed to durably prevent these "safety-removal" attacks. A new defensive strategy, termed "Fool's Gold" or decoy hardening, shifts the paradigm from prevention to deception. This defense concedes that attackers can strip away refusal mechanisms but then poisons the utility of such an attack. Once the safety alignment is removed, the model is trained to confidently and fluently generate decoy answers to hazardous operational requests. Crucially, the critical elements within these decoy responses are falsified, rendering the information useless or dangerous to the attacker. The decoys are trained within a differentiable simulation of the attack, only activating in the attacked state, while a "refusal pin" and "benign leash" maintain original behavior in the clean state. Evaluated across seven models, Fool's Gold successfully generated decoys for a significant portion of attacked-state responses, making it difficult for attackers to distinguish falsified answers from correct ones. This epistemic defense aims to undermine trust in the model's outputs post-attack, particularly for chemical and biological hazards, though it does not address in-context jailbreaks.