EmoAttack: Emotion-to-Image Diffusion Models for Emotional Backdoor Generation
Text-to-image diffusion models can generate realistic images based on textual inputs, enabling users to convey their opinions visually through language. Meanwhile, within language, emotion plays a crucial role in expressing personal opinions in our daily lives and the inclusion of maliciously negative content can lead users astray, exacerbating negative emotions. Recognizing the success of diffusion models and the significance of emotion, we investigate a previously overlooked risk associated with text-to-image diffusion models, that is, utilizing emotion in the input texts to introduce negative content and provoke unfavorable emotions in users. Specifically, we identify a new backdoor attack, i.e., emotion-aware backdoor attack (EmoAttack), which introduces malicious negative content triggered by emotional texts during image generation. We formulate such an attack as a diffusion personalization problem to avoid extensive model retraining and propose the EmoBooth. Unlike existing personalization methods, our approach fine-tunes a pre-trained diffusion model by establishing a mapping between a cluster of emotional words and a given reference image containing malicious negative content. To validate the effectiveness of our method, we built a dataset and conducted extensive analysis and discussion about its effectiveness. Given consumers' widespread use of diffusion models, uncovering this threat is critical for society.
Code (0)
등록된 구현이 없습니다.
Tasks
Backdoor AttackDiffusion PersonalizationImage GenerationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
EmoAttack: Utilizing Emotional Voice Conversion for Speech Backdoor Attacks on Deep Speech Classification Models
Deep speech classification tasks, mainly including keyword spotting and speaker verification, play a crucial role in speech-based human-computer interaction. Recently, the security of these technologies has been demonstr…
AttributeBackdoor AttackKeyword SpottingSpeaker Verification+1When Emotion Becomes Trigger: Emotion-style dynamic Backdoor Attack Parasitising Large Language Models
Data-poisoning backdoors pose a practical threat to the fine-tuning of large language models (LLMs). Most existing attacks bind an attacker-selected behavior to fixed tokens, phrases, scenarios, or syntactic structures. …
Make Me Happier: Evoking Emotions Through Image Diffusion Models
Despite the rapid progress in image generation, emotional image editing remains under-explored. The semantics, context, and structure of an image can evoke emotional responses, making emotional image editing techniques v…
Image GenerationRegressor-Guided Image Editing Regulates Emotional Response to Reduce Online Engagement
Emotions are known to mediate the relationship between users' content consumption and their online engagement, with heightened emotional intensity leading to increased engagement. Building on this insight, we propose thr…
Generative Adversarial NetworkMUSE: Manipulating Unified Framework for Synthesizing Emotions in Images via Test-Time Optimization
Images evoke emotions that profoundly influence perception, often prioritized over content. Current Image Emotional Synthesis (IES) approaches artificially separate generation and editing tasks, creating inefficiencies a…
Semantic Similarity