Backdoor Attacks on Multi-modal Contrastive Learning
Contrastive learning has become a leading self- supervised approach to representation learning across domains, including vision, multimodal settings, graphs, and federated learning. However, recent studies have shown that contrastive learning is susceptible to backdoor and data poisoning attacks. In these attacks, adversaries can manipulate pretraining data or model updates to insert hidden malicious behavior. This paper offers a thorough and comparative review of backdoor attacks in contrastive learning. It analyzes threat models, attack methods, target domains, and available defenses. We summarize recent advancements in this area, underline the specific vulnerabilities inherent to contrastive learning, and discuss the challenges and future research directions. Our findings have significant implications for the secure deployment of systems in industrial and distributed environments.
Code (0)
등록된 구현이 없습니다.
Tasks
Representation LearningContrastive LearningFederated LearningSimilar Papers 제목 키워드 기반
CleanCLIP: Mitigating Data Poisoning Attacks in Multimodal Contrastive Learning
Multimodal contrastive pretraining has been used to train multimodal representation models, such as CLIP, on large amounts of paired image-text data. However, previous studies have revealed that such models are vulnerabl…
Backdoor AttackContrastive LearningData PoisoningBadCLIP: Dual-Embedding Guided Backdoor Attack on Multimodal Contrastive Learning
Studying backdoor attacks is valuable for model copyright protection and enhancing defenses. While existing backdoor attacks have successfully infected multimodal contrastive learning models such as CLIP, they can be eas…
Backdoor AttackContrastive LearningRobust Defense Strategies for Multimodal Contrastive Learning: Efficient Fine-tuning Against Backdoor Attacks
The advent of multimodal deep learning models, such as CLIP, has unlocked new frontiers in a wide range of applications, from image-text understanding to classification tasks. However, these models are not safe for adver…
Multimodal Deep LearningContrastive LearningImage SegmentationRobust Contrastive Language-Image Pre-training against Data Poisoning and Backdoor Attacks
Contrastive vision-language representation learning has achieved state-of-the-art performance for zero-shot classification, by learning from millions of image-caption pairs crawled from the internet. However, the massive…
Backdoor AttackData PoisoningRepresentation Learningzero-shot-classification+1Robust Contrastive Language-Image Pretraining against Data Poisoning and Backdoor Attacks
Contrastive vision-language representation learning has achieved state-of-the-art performance for zero-shot classification, by learning from millions of image-caption pairs crawled from the internet. However, the massive…