Rethinking Training for De-biasing Text-to-Image Generation: Unlocking the Potential of Stable Diffusion
Recent advancements in text-to-image models, such as Stable Diffusion, show significant demographic biases. Existing de-biasing techniques rely heavily on additional training, which imposes high computational costs and risks of compromising core image generation functionality. This hinders them from being widely adopted to real-world applications. In this paper, we explore Stable Diffusion's overlooked potential to reduce bias without requiring additional training. Through our analysis, we uncover that initial noises associated with minority attributes form "minority regions" rather than scattered. We view these "minority regions" as opportunities in SD to reduce bias. To unlock the potential, we propose a novel de-biasing method called 'weak guidance,' carefully designed to guide a random noise to the minority regions without compromising semantic integrity. Through analysis and experiments on various versions of SD, we demonstrate that our proposed approach effectively reduces bias without additional training, achieving both efficiency and preservation of core image generation functionality.
Code (0)
등록된 구현이 없습니다.
Tasks
FairnessImage GenerationText to Image GenerationText-to-Image GenerationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Bias Is a Subspace, Not a Coordinate: A Geometric Rethinking of Post-hoc Debiasing in Vision-Language Models
Vision-Language Models (VLMs) have become indispensable for multimodal reasoning, yet their representations often encode and amplify demographic biases, resulting in biased associations and misaligned predictions in down…
Multimodal ReasoningImage GenerationImage RetrievalBendVLM: Test-Time Debiasing of Vision-Language Embeddings
Vision-language model (VLM) embeddings have been shown to encode biases present in their training data, such as societal biases that prescribe negative characteristics to members of various racial and gender identities. …
AttributeImage GenerationLanguage ModelingLanguage ModellingA Closed-Form Solution for Debiasing Vision-Language Models with Utility Guarantees Across Modalities and Tasks
While Vision-Language Models (VLMs) have achieved remarkable performance across diverse downstream tasks, recent studies have shown that they can inherit social biases from the training data and further propagate them in…
Zero-Shot Image ClassificationText-to-Image GenerationImage RetrievalTowards Real-world Debiasing: Rethinking Evaluation, Challenge, and Solution
Spurious correlations in training data significantly hinder the generalization capability of machine learning models when faced with distribution shifts, leading to the proposition of numberous debiasing methods. However…
A Unified Debiasing Approach for Vision-Language Models across Modalities and Tasks
Recent advancements in Vision-Language Models (VLMs) have enabled complex multimodal tasks by processing text and image data simultaneously, significantly enhancing the field of artificial intelligence. However, these mo…
FairnessImage CaptioningImage GenerationImage Retrieval+5