A Novel Latent-Class Attack and its Detection by Class Subspace Orthogonalization
Deep learning, which in general relies on voluminous amounts of training data, is vulnerable to data poisoning attacks, including error-generic attacks and backdoors (Trojans). In this work, we propose a new data poisoning attack we dub a latent class attack. Here, all poisoned examples are from a class that is novel (unknown) for the given classification domain and are mislabeled to one of the known classes (the target class) of the domain, so that the model learns to recognize the novel class as a sub-class of the target class. Such attacks could be used e.g. to defeat AI-based access control systems, or could cause a "foe" to be classified as a "friend". We also propose a post-training defense to detect this attack, without any access to the training set. This detection approach builds on "class subspace orthogonalization" (CSO), a plug-and-play paradigm demonstrated to improve existing backdoor detectors. Here, CSO is used to seek an input (a putative unknown class instance) whose internal representation is not aligned with any of the known classes, and yet which is classified with confidence to one of these classes. Finally, specific to image classification domains, we propose a method for visualizing the estimated unknown class instance, providing explainability to our latent class detections.
Code (0)
등록된 구현이 없습니다.
Tasks
Image ClassificationSimilar Papers 제목 키워드 기반
Shared Latent Structures Enable Unified Backdoor Detection and Mitigation in LLMs
Backdoor attacks in large language models (LLMs) are often treated as isolated trigger-response failures, motivating defenses tailored to specific triggers or behaviors. We show this view is incomplete. Across diverse ba…
Improving the Sensitivity of Backdoor Detectors via Class Subspace Orthogonalization
Most post-training backdoor detection methods rely on attacked models exhibiting extreme outlier detection statistics for the target class of an attack, compared to non-target classes. However, these approaches may fail:…
Outlier DetectionPiDAn: A Coherence Optimization Approach for Backdoor Attack Detection and Mitigation in Deep Neural Networks
Backdoor attacks impose a new threat in Deep Neural Networks (DNNs), where a backdoor is inserted into the neural network by poisoning the training dataset, misclassifying inputs that contain the adversary trigger. The m…
Anomaly DetectionBackdoor AttackReverse Engineering $\ell_p$ attacks: A block-sparse optimization approach with recovery guarantees
Deep neural network-based classifiers have been shown to be vulnerable to imperceptible perturbations to their input, such as $\ell_p$-bounded norm adversarial attacks. This has motivated the development of many defense …
Detection of Adversarial Attacks and Characterization of Adversarial Subspace
Adversarial attacks have always been a serious threat for any data-driven model. In this paper, we explore subspaces of adversarial examples in unitary vector domain, and we propose a novel detector for defending our mod…
BenchmarkingEnvironmental Sound ClassificationregressionSound Classification