paper-with-me

홈 › Papers

Continuous Concepts Removal in Text-to-image Diffusion Models

2024-11-30 · Tingxu Han, Weisong Sun, Yanrong Hu, Chunrong Fang, Yonglong Zhang, Shiqing Ma, Tao Zheng, Zhenyu Chen, Zhenting Wang

Text-to-image diffusion models have shown an impressive ability to generate high-quality images from input textual descriptions. However, concerns have been raised about the potential for these models to create content that infringes on copyrights or depicts disturbing subject matter. Removing specific concepts from these models is a promising potential solution to this problem. However, existing methods for concept removal do not work well in practical but challenging scenarios where concepts need to be continuously removed. Specifically, these methods lead to poor alignment between the text prompts and the generated image after the continuous removal process. To address this issue, we propose a novel approach called CCRT that includes a designed knowledge distillation paradigm. It constrains the text-image alignment behavior during the continuous concept removal process by using a set of text prompts generated through our genetic algorithm, which employs a designed fuzzing strategy. We conduct extensive experiments involving the removal of various concepts. The results evaluated through both algorithmic metrics and human studies demonstrate that our CCRT can effectively remove the targeted concepts in a continuous manner while maintaining the high generation quality (e.g., text-image alignment) of the model.

📄 PDF Abstract BibTeX arXiv:2412.00580

Code (0)

등록된 구현이 없습니다.

Tasks

Knowledge Distillation

Methods 이 논문이 사용한 방법론

Diffusion Diffusion models generate samples by gradually removing noise from a signal, and their training objective can be expressed as a reweighted variational lower-bound…
SET Dynamic Sparse Training method where weight mask is updated randomly periodically
Knowledge Distillation A very simple way to improve the performance of almost any machine learning algorithm is to train many different models on the same data and then to average their predictions.…

Similar Papers 제목 키워드 기반

Six-CD: Benchmarking Concept Removals for Benign Text-to-image Diffusion Models

2024-06-21 · Jie Ren, Kangrui Chen, Yingqian Cui, Shenglai Zeng 외

Text-to-image (T2I) diffusion models have shown exceptional capabilities in generating images that closely correspond to textual prompts. However, the advancement of T2I diffusion models presents significant risks, as th…

Benchmarking

Six-CD: Benchmarking Concept Removals for Text-to-image Diffusion Models

2025-01-01 · CVPR 2025 1 · Jie Ren, Kangrui Chen, Yingqian Cui, Shenglai Zeng 외

Text-to-image (T2I) diffusion models have shown exceptional capabilities in generating images that closely correspond to textual prompts. However, the advancement of T2I diffusion models presents significant risks, a…

Benchmarking

Implicit Concept Removal of Diffusion Models

2023-10-09 · Zhili Liu, Kai Chen, Yifan Zhang, Jianhua Han 외

Text-to-image (T2I) diffusion models often inadvertently generate unwanted concepts such as watermarks and unsafe images. These concepts, termed as the "implicit concepts", could be unintentionally learned during trainin…

FADE: Adversarial Concept Erasure in Flow Models

2025-07-16 · Zixuan Fu, Yan Ren, Finn Carter, Chenyue Wang 외

Diffusion models have demonstrated remarkable image generation capabilities, but also pose risks in privacy and fairness by memorizing sensitive concepts or perpetuating biases. We propose a novel \textbf{concept erasure…

FairnessImage Generation

Iterative Token Evaluation and Refinement for Real-World Super-Resolution

2023-12-09 · Chaofeng Chen, Shangchen Zhou, Liang Liao, HaoNing Wu 외

Real-world image super-resolution (RWSR) is a long-standing problem as low-quality (LQ) images often have complex and unidentified degradations. Existing methods such as Generative Adversarial Networks (GANs) or continuo…

Image Super-ResolutionSuper-ResolutionTexture Synthesis