paper-with-me

Papers

C^2 ATTACK: Towards Representation Backdoor on CLIP via Concept Confusion

2025-03-12 · Lijie Hu, Junchi Liao, Weimin Lyu, Shaopeng Fu, Tianhao Huang, Shu Yang, Guimin Hu, Di Wang

Backdoor attacks pose a significant threat to deep learning models, enabling adversaries to embed hidden triggers that manipulate the behavior of the model during inference. Traditional backdoor attacks typically rely on inserting explicit triggers (e.g., external patches, or perturbations) into input data, but they often struggle to evade existing defense mechanisms. To address this limitation, we investigate backdoor attacks through the lens of the reasoning process in deep learning systems, drawing insights from interpretable AI. We conceptualize backdoor activation as the manipulation of learned concepts within the model's latent representations. Thus, existing attacks can be seen as implicit manipulations of these activated concepts during inference. This raises interesting questions: why not manipulate the concepts explicitly? This idea leads to our novel backdoor attack framework, Concept Confusion Attack (C^2 ATTACK), which leverages internal concepts in the model's reasoning as "triggers" without introducing explicit external modifications. By avoiding the use of real triggers and directly activating or deactivating specific concepts in latent spaces, our approach enhances stealth, making detection by existing defenses significantly harder. Using CLIP as a case study, experimental results demonstrate the effectiveness of C^2 ATTACK, achieving high attack success rates while maintaining robustness against advanced defenses.

📄 PDF Abstract BibTeX arXiv:2503.09095

Code (0)

등록된 구현이 없습니다.

Tasks

Backdoor Attack

Methods 이 논문이 사용한 방법론

CLIP Contrastive Language-Image Pre-training (CLIP), consisting of a simplified version of ConVIRT trained from scratch, is an efficient method of image representation learning…

Similar Papers 제목 키워드 기반

CleanCLIP: Mitigating Data Poisoning Attacks in Multimodal Contrastive Learning

2023-03-06 · ICCV 2023 1 · Hritik Bansal, Nishad Singhi, Yu Yang, Fan Yin 외

Multimodal contrastive pretraining has been used to train multimodal representation models, such as CLIP, on large amounts of paired image-text data. However, previous studies have revealed that such models are vulnerabl…

Backdoor AttackContrastive LearningData Poisoning

Detecting Backdoor Samples in Contrastive Language Image Pretraining

2025-02-03 · Hanxun Huang, Sarah Erfani, Yige Li, Xingjun Ma 외

Contrastive language-image pretraining (CLIP) has been found to be vulnerable to poisoning backdoor attacks where the adversary can achieve an almost perfect attack success rate on CLIP models by poisoning only 0.01\% of…

Better Safe than Sorry: Pre-training CLIP against Targeted Data Poisoning and Backdoor Attacks

2023-10-05 · Wenhan Yang, Jingdong Gao, Baharan Mirzasoleiman

Contrastive Language-Image Pre-training (CLIP) on large image-caption datasets has achieved remarkable success in zero-shot classification and enabled transferability to new domains. However, CLIP is extremely more vulne…

Contrastive LearningData Poisoningzero-shot-classificationZero-Shot Learning

Robust Contrastive Language-Image Pre-training against Data Poisoning and Backdoor Attacks

2023-03-13 · Wenhan Yang, Jingdong Gao, Baharan Mirzasoleiman

Contrastive vision-language representation learning has achieved state-of-the-art performance for zero-shot classification, by learning from millions of image-caption pairs crawled from the internet. However, the massive…

Backdoor AttackData PoisoningRepresentation Learningzero-shot-classification+1

Robust Contrastive Language-Image Pretraining against Data Poisoning and Backdoor Attacks

2023-09-21 · NeurIPS 2023 11

Contrastive vision-language representation learning has achieved state-of-the-art performance for zero-shot classification, by learning from millions of image-caption pairs crawled from the internet. However, the massive…