On Conditioning the Input Noise for Controlled Image Generation with Diffusion Models
Conditional image generation has paved the way for several breakthroughs in image editing, generating stock photos and 3-D object generation. This continues to be a significant area of interest with the rise of new state-of-the-art methods that are based on diffusion models. However, diffusion models provide very little control over the generated image, which led to subsequent works exploring techniques like classifier guidance, that provides a way to trade off diversity with fidelity. In this work, we explore techniques to condition diffusion models with carefully crafted input noise artifacts. This allows generation of images conditioned on semantic attributes. This is different from existing approaches that input Gaussian noise and further introduce conditioning at the diffusion model's inference step. Our experiments over several examples and conditional settings show the potential of our approach.
Code (0)
등록된 구현이 없습니다.
Tasks
Conditional Image GenerationDiversityImage GenerationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Self-supervision via Controlled Transformation and Unpaired Self-conditioning for Low-light Image Enhancement
Real-world low-light images captured by imaging devices suffer from poor visibility and require a domain-specific enhancement to produce artifact-free outputs that reveal details. In this paper, we propose an unpaired lo…
DenoisingImage EnhancementLow-Light Image EnhancementRepFusion: Leveraging Multimodal Priors for Denoising in Representation Space
Large language models (LLMs) are widely used in text-to-image (T2I) systems, but they are typically limited to text encoding, while denoising is handled by newly trained generative backbones. The emergence of representat…
Infinite-Homography as Robust Conditioning for Camera-Controlled Video Generation
Recent progress in video diffusion models has spurred growing interest in camera-controlled novel-view video generation for dynamic scenes, aiming to provide creators with cinematic camera control capabilities in post-pr…
Data AugmentationDepth EstimationVideo GenerationDreamCache: Finetuning-Free Lightweight Personalized Image Generation via Feature Caching
Personalized image generation requires text-to-image generative models that capture the core features of a reference subject to allow for controlled generation across different contexts. Existing methods face challenges …
Image GenerationPersonalized Image GenerationDistribution Conditional Denoising: A Flexible Discriminative Image Denoiser
A flexible discriminative image denoiser is introduced in which multi-task learning methods are applied to a densoising FCN based on U-Net. The activations of the U-Net model are modified by affine transforms that are a …
DenoisingMulti-Task Learning