TestifAI Offers Efficient Tomography-Based Robustness Testing for Deep Learning

As AI systems, particularly those based on deep learning, are increasingly deployed in safety-critical sectors like autonomous driving, ensuring their correct behavior through rigorous testing becomes paramount. Traditional robustness testing involves thousands of inferences to verify model stability under bounded input perturbations. However, existing frameworks struggle to systematically explore and summarize robustness across the vast, combinatorial space of possible perturbations.Researchers propose TestifAI, a novel deep learning testing framework designed for efficient and accurate robustness estimation against combinations of perturbations. TestifAI allows users to define operational conditions as structured spaces of semantic input perturbations (e.g., image blur, brightness, zoom) with discrete severity levels. Users can then query model robustness for any specific combination of these perturbations.To achieve both efficiency and accuracy, TestifAI introduces "partial model tomography." This innovative approach reconstructs model behavior in a multi-perturbation space by performing tests on only a small number of perturbations (lower-order projections). For instance, to estimate robustness against three or more perturbations, TestifAI trains an auxiliary model using results from tests involving only one or two perturbations. Experiments across five image and language classification tasks show that TestifAI can predict higher-order test outcomes with less than 7% error, while reducing the number of inferences by 60-80%.