Harm Amplification in Text-to-Image Models
Text-to-image (T2I) models have emerged as a significant advancement in generative AI; however, there exist safety concerns regarding their potential to produce harmful image outputs even when users input seemingly safe prompts. This phenomenon, where T2I models generate harmful representations that were not explicit in the input prompt, poses a potentially greater risk than adversarial prompts, leaving users unintentionally exposed to harms. Our paper addresses this issue by formalizing a definition for this phenomenon which we term harm amplification. We further contribute to the field by developing a framework of methodologies to quantify harm amplification in which we consider the harm of the model output in the context of user input. We then empirically examine how to apply these different methodologies to simulate real-world deployment scenarios including a quantification of disparate impacts across genders resulting from harm amplification. Together, our work aims to offer researchers tools to comprehensively address safety challenges in T2I systems and contribute to the responsible deployment of generative AI models.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
HARP: Measuring Harm Amplification in Multi-Agent LLM Systems
Multi-agent LLM systems decompose workflows across agents, tools, shared context, memory, and decision gates. This modularity improves interpretability, but creates a propagation risk: a bounded perturbation to one compo…
Men Also Do Laundry: Multi-Attribute Bias Amplification
As computer vision systems become more widely deployed, there is increasing concern from both the research community and the public that these systems are not only reproducing but amplifying harmful social biases. The ph…
AttributeAmplification Effects in Test-Time Reinforcement Learning: Safety and Reasoning Vulnerabilities
Test-time training (TTT) has recently emerged as a promising method to improve the reasoning abilities of large language models (LLMs), in which the model directly learns from test data without access to labels. However,…
Reinforcement LearningInvestigating and Alleviating Harm Amplification in LLM Interactions
Large language models (LLMs) can serve as helpful assistants, yet they can equally function as harm amplifiers that enable malicious users to achieve harmful outcomes beyond their capabilities through extended interactio…
Quasiperiodic Disturbance Observer for Wideband Harmonic Suppression
Periodic disturbances composed of harmonics typically occur during periodic operations, impairing performance of mechanical and electrical systems. To improve the performance, control of periodic-disturbance suppression …