Contrastive Denoising Score for Text-guided Latent Diffusion Image Editing
With the remarkable advent of text-to-image diffusion models, image editing methods have become more diverse and continue to evolve. A promising recent approach in this realm is Delta Denoising Score (DDS) - an image editing technique based on Score Distillation Sampling (SDS) framework that leverages the rich generative prior of text-to-image diffusion models. However, relying solely on the difference between scoring functions is insufficient for preserving specific structural elements from the original image, a crucial aspect of image editing. To address this, here we present an embarrassingly simple yet very powerful modification of DDS, called Contrastive Denoising Score (CDS), for latent diffusion models (LDM). Inspired by the similarities and differences between DDS and the contrastive learning for unpaired image-to-image translation(CUT), we introduce a straightforward approach using CUT loss within the DDS framework. Rather than employing auxiliary networks as in the original CUT approach, we leverage the intermediate features of LDM, specifically those from the self-attention layers, which possesses rich spatial information. Our approach enables zero-shot image-to-image translation and neural radiance field (NeRF) editing, achieving structural correspondence between the input and output while maintaining content controllability. Qualitative results and comparisons demonstrates the effectiveness of our proposed method. Project page: https://hyelinnam.github.io/CDS/
Code (1)
Tasks
Contrastive LearningDenoisingImage-to-Image TranslationNeRFTranslationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Sample-Adaptive Latent Rewards for Uncertainty-Guided Diffusion Post-Training
Latent reward models can supervise visual diffusion models without decoding intermediate states into pixel space. This makes alignment with human preferences more efficient. However, existing latent reward models output …
Diffusion-Based Representation Learning
Diffusion-based methods represented as stochastic differential equations on a continuous-time domain have recently proven successful as a non-adversarial generative model. Training such models relies on denoising score m…
Denoisingimage-classificationImage ClassificationRepresentation Learning+1Similarity-Guided Diffusion for Contrastive Sequential Recommendation
In sequential recommendation systems, data augmentation and contrastive learning techniques have recently been introduced using diffusion models to achieve robust representation learning. However, most of the existing ap…
Contrastive LearningData AugmentationDenoisingRecommendation Systems+2Single-Reference Text-to-Image Manipulation with Dual Contrastive Denoising Score
Large-scale text-to-image generative models have shown remarkable ability to synthesize diverse and high-quality images. However, it is still challenging to directly apply these models for editing real images for two rea…
Image-to-Image TranslationContrastive LearningImage ManipulationImage EditingContrastive Blind Denoising Autoencoder for Real-Time Denoising of Industrial IoT Sensor Data
In an industrial IoT setting, ensuring the quality of sensor data is a must when data-driven algorithms operate on the upper layers of the control system. Unfortunately, the common place in industrial facilities is to fi…
DenoisingSelf-Supervised LearningTime SeriesTime Series Analysis