paper-with-me

홈 › Papers

How to Blend Concepts in Diffusion Models

2024-07-19 · Lorenzo Olearo, Giorgio Longari, Simone Melzi, Alessandro Raganato, Rafael Peñaloza

For the last decade, there has been a push to use multi-dimensional (latent) spaces to represent concepts; and yet how to manipulate these concepts or reason with them remains largely unclear. Some recent methods exploit multiple latent representations and their connection, making this research question even more entangled. Our goal is to understand how operations in the latent space affect the underlying concepts. To that end, we explore the task of concept blending through diffusion models. Diffusion models are based on a connection between a latent representation of textual prompts and a latent space that enables image reconstruction and generation. This task allows us to try different text-based combination strategies, and evaluate easily through a visual analysis. Our conclusion is that concept blending through space manipulation is possible, although the best strategy depends on the context of the blend.

📄 PDF Abstract BibTeX arXiv:2407.14280

Code (1)

LorenzoOlearo/blending-diffusion-models 공식 구현 pytorch

Tasks

Image Reconstruction

Methods 이 논문이 사용한 방법론

Diffusion Diffusion models generate samples by gradually removing noise from a signal, and their training objective can be expressed as a reweighted variational lower-bound…

Similar Papers 제목 키워드 기반

Adaptive Auxiliary Prompt Blending for Target-Faithful Diffusion Generation

2026-03-19 · Kwanyoung Lee, SeungJu Cha, Yebin Ahn, Hyunwoo Oh 외 arxiv

Diffusion-based text-to-image (T2I) models have made remarkable progress in generating photorealistic and semantically rich images. However, when the target concepts lie in low-density regions of the training distributio…

Image Editing

TweedieMix: Improving Multi-Concept Fusion for Diffusion-based Image/Video Generation

2024-10-08 · Gihyun Kwon, Jong Chul Ye

Despite significant advancements in customizing text-to-image and video generation models, generating images and videos that effectively integrate multiple personalized concepts remains a challenging task. To address thi…

Video Generation

LatexBlend: Scaling Multi-concept Customized Generation with Latent Textual Blending

2025-03-10 · CVPR 2025 1 · Jian Jin, Zhenbo Yu, Yang shen, ZhenYong Fu 외

Customized text-to-image generation renders user-specified concepts into novel contexts based on textual prompts. Scaling the number of concepts in customized generation meets a broader demand for user creation, whereas …

Computational EfficiencyDenoisingImage GenerationText to Image Generation+1

MeshUp: Multi-Target Mesh Deformation via Blended Score Distillation

2024-08-27 · Hyunwoo Kim, Itai Lang, Noam Aigerman, Thibault Groueix 외

We propose MeshUp, a technique that deforms a 3D mesh towards multiple target concepts, and intuitively controls the region where each concept is expressed. Conveniently, the concepts can be defined as either text querie…

Denoising

ConceptSplit: Decoupled Multi-Concept Personalization of Diffusion Models via Token-wise Adaptation and Attention Disentanglement

2025-10-06 · Habin Lim, Yeongseob Won, Juwon Seo, Gyeong-Moon Park arxiv

In recent years, multi-concept personalization for text-to-image (T2I) diffusion models to represent several subjects in an image has gained much more attention. The main challenge of this task is "concept mixing", where…