Lookahead Lemmas Boost Neural Network Verification Performance

Neural network verification, a crucial step for ensuring the reliability and safety of AI systems, primarily relies on branch-and-bound procedures. This new research proposes an innovative inprocessing framework designed to enhance these verification processes through the use of "lookahead lemmas." The framework operates by employing a lookahead procedure to generate new lemmas, specifically focusing on the unstable phases of ReLU activation functions. These derived lemmas are then compiled into an implication graph. This graph serves as a powerful tool to prune the search space, effectively reducing the complexity and computational burden of the verification task. The researchers instantiated this framework within two leading neural network verifiers, Marabou and α-β-CROWN, demonstrating significant performance improvements, including proving up to 34% more instances unsatisfiable.