LiST Improves Neural Network Robustness and Calibration

Neural networks often struggle to simultaneously achieve high accuracy, robustness, and calibration, which are all critical for reliable AI systems. Lipschitz-constrained models offer robustness guarantees, but selecting the optimal Lipschitz constraint (L) to balance accuracy and robustness has been a manual and underexplored process, especially regarding calibration. This research highlights a theoretical and empirical connection between the enforced Lipschitz constraint and Temperature Scaling, a leading calibration method. The study found that for any given training scheme, a specific Lipschitz value (L*) exists that naturally yields an out-of-the-box calibrated network. This insight suggests that calibration can serve as a principled criterion for selecting an optimal operating point on the accuracy-robustness Pareto front. Building on this, the researchers propose Lipschitz Scaling Training (LiST), a new training paradigm. LiST iteratively adjusts the global Lipschitz constant to reach this optimal operating point. By incorporating a margin parameter in the training loss, LiST can construct a fully calibrated Pareto front, allowing users to navigate the accuracy-robustness trade-off while ensuring calibration throughout. Upon convergence, LiST also enables the re-integration of calibration data into training, boosting sample efficiency without compromising calibration. Validated on CIFAR-10/100 and Tiny-ImageNet, LiST demonstrates competitive accuracy and robustness compared to other baselines, all while remaining calibrated without additional post-processing.