Red-Teaming the Stable Diffusion Safety Filter
Stable Diffusion is a recent open-source image generation model comparable to proprietary models such as DALLE, Imagen, or Parti. Stable Diffusion comes with a safety filter that aims to prevent generating explicit images. Unfortunately, the filter is obfuscated and poorly documented. This makes it hard for users to prevent misuse in their applications, and to understand the filter's limitations and improve it. We first show that it is easy to generate disturbing content that bypasses the safety filter. We then reverse-engineer the filter and find that while it aims to prevent sexual content, it ignores violence, gore, and other similarly disturbing content. Based on our analysis, we argue safety measures in future model releases should strive to be fully open and properly documented to stimulate security contributions from the community.
Code (0)
등록된 구현이 없습니다.
Tasks
Image GenerationRed TeamingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Prompting4Debugging: Red-Teaming Text-to-Image Diffusion Models by Finding Problematic Prompts
Text-to-image diffusion models, e.g. Stable Diffusion (SD), lately have shown remarkable ability in high-quality content generation, and become one of the representatives for the recent wave of transformative AI. Neverth…
Red TeamingText-to-Image Generationtrustable and focussed LLM generated contentGenBreak: Red Teaming Text-to-Image Generators Using Large Language Models
Text-to-image (T2I) models such as Stable Diffusion have advanced rapidly and are now widely used in content creation. However, these models can be misused to generate harmful content, including nudity or violence, posin…
Large Language ModelRed TeamingRing-A-Bell! How Reliable are Concept Removal Methods for Diffusion Models?
Diffusion models for text-to-image (T2I) synthesis, such as Stable Diffusion (SD), have recently demonstrated exceptional capabilities for generating high-quality content. However, this progress has raised several concer…
Red TeamingDREAM: Scalable Red Teaming for Text-to-Image Generative Systems via Distribution Modeling
Despite the integration of safety alignment and external filters, text-to-image (T2I) generative systems are still susceptible to producing harmful content, such as sexual or violent imagery. This raises serious concerns…
Red TeamingFLIRT: Feedback Loop In-context Red Teaming
Warning: this paper contains content that may be inappropriate or offensive. As generative models become available for public use in various applications, testing and analyzing vulnerabilities of these models has become …
In-Context LearningRed TeamingResponse Generation