MMA-Diffusion: MultiModal Attack on Diffusion Models
In recent years, Text-to-Image (T2I) models have seen remarkable advancements, gaining widespread adoption. However, this progress has inadvertently opened avenues for potential misuse, particularly in generating inappropriate or Not-Safe-For-Work (NSFW) content. Our work introduces MMA-Diffusion, a framework that presents a significant and realistic threat to the security of T2I models by effectively circumventing current defensive measures in both open-source models and commercial online services. Unlike previous approaches, MMA-Diffusion leverages both textual and visual modalities to bypass safeguards like prompt filters and post-hoc safety checkers, thus exposing and highlighting the vulnerabilities in existing defense mechanisms.
Code (2)
Similar Papers 제목 키워드 기반
Adversarial-Guided Diffusion for Multimodal LLM Attacks
This paper addresses the challenge of generating adversarial image using a diffusion model to deceive multimodal large language models (MLLMs) into generating the targeted responses, while avoiding significant distortion…
Adversarial AttackText-to-Image Diffusion Models can be Easily Backdoored through Multimodal Data Poisoning
With the help of conditioning mechanisms, the state-of-the-art diffusion models have achieved tremendous success in guided image generation, particularly in text-to-image synthesis. To gain a better understanding of the …
Backdoor Attackbackdoor defenseData PoisoningImage GenerationWhen One Modality Rules Them All: Backdoor Modality Collapse in Multimodal Diffusion Models
While diffusion models have revolutionized visual content generation, their rapid adoption has underscored the critical need to investigate vulnerabilities, e.g., to backdoor attacks. In multimodal diffusion models, it i…
CoDefend: Cross-Modal Collaborative Defense via Diffusion Purification and Prompt Optimization
Multimodal Large Language Models (MLLMs) have achieved remarkable success in tasks such as image captioning, visual question answering, and cross-modal reasoning by integrating visual and textual modalities. However, the…
Visual Question AnsweringImage CaptioningCBV: Clean-label Backdoor Attacks on Vision Language Models via Diffusion Models
Vision-Language Models (VLMs) have achieved remarkable success in tasks such as image captioning and visual question answering (VQA). However, as their applications become increasingly widespread, recent studies have rev…
Visual Question AnsweringImage Captioning