New Method Boosts LLM Adversarial Training Efficiency

Adversarial training is a critical defense mechanism against attacks on large language models (LLMs), but its computational demands are often prohibitive. Existing methods, such as latent adversarial training (LAT), still incur substantial costs. This new work introduces a dual-pronged approach to enhance LAT's efficiency. On the defense side, the researchers explore representation fine-tuning (ReFT) within LAT, identifying and addressing potential mismatches between ReFT application and attack vectors. For the attack side, they propose extracting only the relevant computational circuits from the LLM to create a lightweight surrogate model. This surrogate model is then used for generating adversarial attacks, bypassing the need for full forward-backward passes through the entire LLM, thereby saving significant computational resources. The proposed strategies are supported by theoretical justifications and empirical evidence, demonstrating an average reduction of 48.1% in per-step adversarial training FLOPs while requiring only a tiny fraction (0.0118%) of trainable parameters compared to standard LAT.