DreamDistribution: Learning Prompt Distribution for Diverse In-distribution Generation
The popularization of Text-to-Image (T2I) diffusion models enables the generation of high-quality images from text descriptions. However, generating diverse customized images with reference visual attributes remains challenging. This work focuses on personalizing T2I diffusion models at a more abstract concept or category level, adapting commonalities from a set of reference images while creating new instances with sufficient variations. We introduce a solution that allows a pretrained T2I diffusion model to learn a set of soft prompts, enabling the generation of novel images by sampling prompts from the learned distribution. These prompts offer text-guided editing capabilities and additional flexibility in controlling variation and mixing between multiple distributions. We also show the adaptability of the learned prompt distribution to other tasks, such as text-to-3D. Finally we demonstrate effectiveness of our approach through quantitative analysis including automatic evaluation and human assessment. Project website: https://briannlongzhao.github.io/DreamDistribution
Code (0)
등록된 구현이 없습니다.
Tasks
Text to 3DMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Robust Length Prediction: A Perspective from Heavy-Tailed Prompt-Conditioned Distributions
Output-length prediction is important for efficient LLM serving, as it directly affects batching, memory reservation, and scheduling. For prompt-only length prediction, most existing methods use a one-shot sampled length…
Learning Probabilistic Prompt for Continual Learning
Continual learning aims to progressively learn from a sequence of tasks, each containing a disjoint subset of classes, while preserving previously learned knowledge. Prompt-based continual learning methods propose to lea…
Continual LearningControlled Training Data Generation with Diffusion Models
We present a method to control a text-to-image generative model to produce training data useful for supervised learning. Unlike previous works that employ an open-loop approach and pre-define prompts to generate new data…
Language ModelingLanguage ModellingImproving the Distributional Alignment of LLMs using Supervision
The ability to accurately align LLMs with diverse population groups on subjective questions would have great value. In this work, we show that adding simple supervision can more consistently improve the alignment of LLM-…
Controlling Distributional Bias in Multi-Round LLM Generation via KL-Optimized Fine-Tuning
While the real world is inherently stochastic, Large Language Models (LLMs) are predominantly evaluated on single-round inference against fixed ground truths. In this work, we shift the lens to distribution alignment: as…
Prompt Engineering