AdaStop Optimizes DNN Testing with Cost-Aware Early Stopping.

Testing deep neural networks (DNNs) typically focuses on identifying inputs likely to reveal faults within a predetermined labeling budget. However, setting this budget is challenging; too little testing risks missing critical failures, while excessive testing incurs unnecessary costs. This research addresses the DNN testing "stopping problem" by framing it as a cost-benefit decision. The proposed framework, named AdaStop, models the testing process where labeling an input incurs a cost, and discovering a fault yields a specific value. AdaStop continuously estimates the marginal rate at which new faults are discovered during testing. The system then intelligently halts the labeling process when this estimated fault discovery rate drops below a predefined threshold, which is calculated as the ratio of labeling cost to fault value. Experiments across various datasets, architectures, and selection strategies demonstrate AdaStop's efficiency, showing that it can uncover 65-84% of faults using only 9-31% of the total labeling budget.