Stealthy and Adjustable Text-Guided Backdoor Attacks on Multimodal Pretrained Models
Multimodal pretrained models are vulnerable to backdoor attacks, yet most existing methods rely on visual or multimodal triggers, which are impractical since visually embedded triggers rarely occur in real-world data. To overcome this limitation, we propose a novel Text-Guided Backdoor (TGB) attack on multimodal pretrained models, where commonly occurring words in textual descriptions serve as backdoor triggers, significantly improving stealthiness and practicality. Furthermore, we introduce visual adversarial perturbations on poisoned samples to modulate the model's learning of textual triggers, enabling a controllable and adjustable TGB attack. Extensive experiments on downstream tasks built upon multimodal pretrained models, including Composed Image Retrieval (CIR) and Visual Question Answering (VQA), demonstrate that TGB achieves practicality and stealthiness with adjustable attack success rates across diverse realistic settings, revealing critical security vulnerabilities in multimodal pretrained models.
Code (0)
등록된 구현이 없습니다.
Tasks
Visual Question AnsweringImage RetrievalSimilar Papers 제목 키워드 기반
Rethink the Evaluation for Attack Strength of Backdoor Attacks in Natural Language Processing
It has been shown that natural language processing (NLP) models are vulnerable to a kind of security threat called the Backdoor Attack, which utilizes a `backdoor trigger' paradigm to mislead the models. The most threate…
Backdoor AttackText ClassificationTowards Stealthy Backdoor Attacks against Speech Recognition via Elements of Sound
Deep neural networks (DNNs) have been widely and successfully adopted and deployed in various applications of speech recognition. Recently, a few works revealed that these models are vulnerable to backdoor attacks, where…
Backdoor Attackspeech-recognitionSpeech RecognitionEnhancing Clean Label Backdoor Attack with Two-phase Specific Triggers
Backdoor attacks threaten Deep Neural Networks (DNNs). Towards stealthiness, researchers propose clean-label backdoor attacks, which require the adversaries not to alter the labels of the poisoned training datasets. Clea…
Backdoor Attackbackdoor defenseVocal Bursts Valence PredictionTASER: Task-Aware Spectral Energy Refine for Backdoor Suppression in UAV Swarms Decentralized Federated Learning
As backdoor attacks in UAV-based decentralized federated learning (DFL) grow increasingly stealthy and sophisticated, existing defenses have likewise escalated in complexity. Yet these defenses, which rely heavily on out…
Federated LearningOutlier DetectionStealthy Backdoor Attack to Real-world Models in Android Apps
Powered by their superior performance, deep neural networks (DNNs) have found widespread applications across various domains. Many deep learning (DL) models are now embedded in mobile apps, making them more accessible to…
Backdoor Attack