New Adversarial Training Method Avoids Input Gradients.

Adversarial training is a crucial technique for making deep neural networks robust against small, imperceptible input perturbations, known as adversarial examples. Traditionally, generating these examples involves iterative computations of input gradients, which can be computationally expensive and slow down the training process. This research proposes a new adversarial training scheme that bypasses the need for input gradient iterations entirely. It achieves this by computing adversarial examples directly from the trained network parameters using a linearization technique called the low-rank Householder expansion (LRHE). This expansion describes the composed affine map of the network and identifies perturbation directions from the activation pattern during a standard forward pass. The proposed method requires only additional forward evaluations, with weight parameters updated via the standard backward pass, effectively eliminating the inner maximization loop of typical min-max adversarial training. This results in a substantial reduction in computational cost, achieving robustness comparable to 3-step PGD adversarial training for small perturbation budgets, making adversarial training more efficient and scalable.