Veriphi System Verifies Neural Networks, Highlights Dataset-Dependent Training

Researchers have developed Veriphi, a GPU-accelerated system designed for verifying neural networks. This system integrates fast adversarial attacks with formal certification methods, specifically alpha,beta-CROWN, to assess model robustness. A key finding from their systematic experiments is that the optimal training methodology for neural networks is highly dependent on the specific dataset being used. The study, conducted on datasets like MNIST and CIFAR-10, revealed that while Interval Bound Propagation (IBP) achieved high certified accuracy on simpler datasets like MNIST, its performance was negligible on more complex ones like CIFAR-10. In contrast, PGD adversarial training proved dominant for CIFAR-10, achieving 94% certification at small perturbations. The Veriphi system also achieved a 5x speedup in verification through attack-guided falsification and successfully scaled to production-size models, challenging the notion that certified training universally outperforms adversarial training.