paper-with-me

홈 › Papers

Training-Free Debiasing of Diffusion Models via CLIP-Guided Denoising Optimization

2026-07-01 · Dain Kim, Jinseo Kim, Sungyong Baik arxiv

Text-to-image diffusion models achieve impressive visual quality, yet demographic bias remains a challenge, as neutral prompts consistently produce stereotypical representations across gender and race. Existing approaches remain limited by costly retraining or by inference-time interventions that often degrade image quality and semantic alignment. We propose Text Embedding Steering (TES), a training-free framework that mitigates demographic bias by directly optimizing conditional text embeddings during the diffusion process. We show that a two-stage strategy - early-stage global alignment followed by iterative denoising-time refinement with CLIP-based feedback - enables stable and controllable attribute steering without modifying model parameters. Extensive experiments on Stable Diffusion demonstrate that TES outperforms existing training-free baselines in fairness while maintaining competitive image quality. These results highlight that inference-time text embedding optimization is a practical and scalable solution for fairness-aware generation in diffusion models.

📄 PDF Abstract BibTeX arXiv:2607.00817

Code (0)

등록된 구현이 없습니다.

Similar Papers 제목 키워드 기반

SegDebias: Test-Time Bias Mitigation for ViT-Based CLIP via Segmentation

2025-11-01 · Fangyu Wu, Yujun Cai arxiv

Vision language models such as CLIP have shown remarkable performance in zero shot classification, but remain susceptible to spurious correlations, where irrelevant visual features influence predictions. Existing debiasi…

LightFair: Towards an Efficient Alternative for Fair T2I Diffusion via Debiasing Pre-trained Text Encoders

2025-09-28 · Boyu Han, Qianqian Xu, Shilong Bao, Zhiyong Yang 외 arxiv

This paper explores a novel lightweight approach LightFair to achieve fair text-to-image diffusion models (T2I DMs) by addressing the adverse effects of the text encoder. Most existing methods either couple different par…

Classifier-guided CLIP Distillation for Unsupervised Multi-label Classification

2025-01-01 · CVPR 2025 1 · Dongseob Kim, Hyunjung Shim

Multi-label classification is crucial for comprehensive image understanding, yet acquiring accurate annotations is challenging and costly. To address this, a recent study suggests exploiting unsupervised multi-label …

ClassificationLanguage ModelingLanguage ModellingMulti-Label Classification+1

SANER: Annotation-free Societal Attribute Neutralizer for Debiasing CLIP

2024-08-19 · Yusuke Hirota, Min-Hung Chen, Chien-Yi Wang, Yuta Nakashima 외

Large-scale vision-language models, such as CLIP, are known to contain societal bias regarding protected attributes (e.g., gender, age). This paper aims to address the problems of societal bias in CLIP. Although previous…

AttributeImage GenerationText-to-Image Generation

PRISM: Reducing Spurious Implicit Biases in Vision-Language Models with LLM-Guided Embedding Projection

2025-07-11 · Mahdiyar Molahasani, Azadeh Motamedi, Michael Greenspan, Il-Min Kim 외 arxiv

We introduce Projection-based Reduction of Implicit Spurious bias in vision-language Models (PRISM), a new data-free and task-agnostic solution for bias mitigation in VLMs like CLIP. VLMs often inherit and amplify biases…