New Framework Evaluates AI Robustness with Minimum-Norm Attacks

Current methods for evaluating adversarial robustness typically rely on predefined attack ensembles, like AutoAttack, at a single perturbation budget (epsilon) and a limited selection of perturbation norms. This approach has significant limitations, as robustness-perturbation curves can intersect, making single-epsilon rankings unstable and providing no guarantee of worst-case performance. This paper proposes a new, unified evaluation framework centered on a comprehensive pool of minimum-norm attacks and the generation of robustness-perturbation curves across L0, L1, L2, and L-infinity norms. The "attack frontier" is defined as the worst-case robustness estimate derived from this attack pool. The framework formalizes evaluation as an "attack frontier approximation problem," constructing optimized subsets of attacks that approach the frontier under a controllable query budget. It also introduces the "defense frontier" and a "Defense Optimality Index" to rank defenses without needing a reference epsilon. On CIFAR-10 and ImageNet, these ensembles consistently match or exceed AutoAttack's performance at various budget tiers, offering a more robust and flexible evaluation alternative.