New Latent-Class Data Poisoning Attack and Detection Method

A new form of data poisoning attack, termed a "latent class attack," has been identified, posing a significant threat to deep learning models. This attack involves introducing data from an entirely novel, unknown class and intentionally mislabeling it as belonging to an existing, known class within the model's training data. The objective is to trick the model into recognizing the novel class as a legitimate subclass of a target category, which could have critical implications for systems like AI-based access control, potentially allowing unauthorized access by misclassifying threats as benign entities. In response to this vulnerability, the researchers have also developed a robust defense mechanism. This post-training detection method, called Class Subspace Orthogonalization (CSO), operates without requiring access to the original training dataset. CSO works by identifying inputs whose internal representations do not align with any established known classes, yet are confidently classified into one of them, indicating a potential latent class attack. For image classification tasks, the proposed method further enhances explainability by providing a way to visualize the estimated unknown class instances, thereby strengthening the diagnostic capabilities against such sophisticated poisoning attempts.