Towards Effective Multi-Label Recognition Attacks via Knowledge Graph Consistency
Many real-world applications of image recognition require multi-label learning, whose goal is to find all labels in an image. Thus, robustness of such systems to adversarial image perturbations is extremely important. However, despite a large body of recent research on adversarial attacks, the scope of the existing works is mainly limited to the multi-class setting, where each image contains a single label. We show that the naive extensions of multi-class attacks to the multi-label setting lead to violating label relationships, modeled by a knowledge graph, and can be detected using a consistency verification scheme. Therefore, we propose a graph-consistent multi-label attack framework, which searches for small image perturbations that lead to misclassifying a desired target set while respecting label hierarchies. By extensive experiments on two datasets and using several multi-label recognition models, we show that our method generates extremely successful attacks that, unlike naive multi-label perturbations, can produce model predictions consistent with the knowledge graph.
Code (0)
등록된 구현이 없습니다.
Tasks
Multi-Label LearningSimilar Papers 제목 키워드 기반
Semantic-Aware Multi-Label Adversarial Attacks
Despite its importance generating attacks for multi label learning (MLL) models has received much less attention compared to multi-class recognition. Attacking an MLL model by optimizing a loss on the target set of l…
Multi-Label LearningLabel Flipping Data Poisoning Attack Against Wearable Human Activity Recognition System
Human Activity Recognition (HAR) is a problem of interpreting sensor data to human movement using an efficient machine learning (ML) approach. The HAR systems rely on data from untrusted users, making them susceptible to…
Activity RecognitionData PoisoningHuman Activity RecognitionRobust Defense Strategies for Multimodal Contrastive Learning: Efficient Fine-tuning Against Backdoor Attacks
The advent of multimodal deep learning models, such as CLIP, has unlocked new frontiers in a wide range of applications, from image-text understanding to classification tasks. However, these models are not safe for adver…
Multimodal Deep LearningContrastive LearningImage SegmentationLook, Listen, and Attack: Backdoor Attacks Against Video Action Recognition
Deep neural networks (DNNs) are vulnerable to a class of attacks called "backdoor attacks", which create an association between a backdoor trigger and a target label the attacker is interested in exploiting. A backdoored…
Action RecognitionTemporal Action LocalizationWicked Oddities: Selectively Poisoning for Effective Clean-Label Backdoor Attacks
Deep neural networks are vulnerable to backdoor attacks, a type of adversarial attack that poisons the training data to manipulate the behavior of models trained on such data. Clean-label attacks are a more stealthy form…
Adversarial AttackFace Recognition