paper-with-me

Papers

ControlEdit: A MultiModal Local Clothing Image Editing Method

2024-09-23 · Di Cheng, Yingjie Shi, ShiXin Sun, JiaFu Zhang, WeiJing Wang, Yu Liu

Multimodal clothing image editing refers to the precise adjustment and modification of clothing images using data such as textual descriptions and visual images as control conditions, which effectively improves the work efficiency of designers and reduces the threshold for user design. In this paper, we propose a new image editing method ControlEdit, which transfers clothing image editing to multimodal-guided local inpainting of clothing images. We address the difficulty of collecting real image datasets by leveraging the self-supervised learning approach. Based on this learning approach, we extend the channels of the feature extraction network to ensure consistent clothing image style before and after editing, and we design an inverse latent loss function to achieve soft control over the content of non-edited areas. In addition, we adopt Blended Latent Diffusion as the sampling method to make the editing boundaries transition naturally and enforce consistency of non-edited area content. Extensive experiments demonstrate that ControlEdit surpasses baseline algorithms in both qualitative and quantitative evaluations.

📄 PDF Abstract BibTeX arXiv:2409.14720

Code (1)

cd123-cd/ControlEdit 공식 구현 pytorch

Tasks

Self-Supervised Learning

Methods 이 논문이 사용한 방법론

Diffusion Diffusion models generate samples by gradually removing noise from a signal, and their training objective can be expressed as a reweighted variational lower-bound…
Inpainting Train a convolutional neural network to generate the contents of an arbitrary image region conditioned on its surroundings.

Similar Papers 제목 키워드 기반

PromptDresser: Improving the Quality and Controllability of Virtual Try-On via Generative Textual Prompt and Prompt-aware Mask

2024-12-22 · Jeongho Kim, Hoiyeong Jin, Sunghyun Park, Jaegul Choo

Recent virtual try-on approaches have advanced by fine-tuning the pre-trained text-to-image diffusion models to leverage their powerful generative ability. However, the use of text prompts in virtual try-on is still unde…

In-Context LearningVirtual Try-on

PISE: Person Image Synthesis and Editing with Decoupled GAN

2021-03-06 · CVPR 2021 1 · Jinsong Zhang, Kun Li, Yu-Kun Lai, Jingyu Yang

Person image synthesis, e.g., pose transfer, is a challenging problem due to large variation and occlusion. Existing methods have difficulties predicting reasonable invisible regions and fail to decouple the shape and st…

Human ParsingImage GenerationPose Transfer

FashionEngine: Interactive 3D Human Generation and Editing via Multimodal Controls

2024-04-02 · Tao Hu, Fangzhou Hong, Zhaoxi Chen, Ziwei Liu

We present FashionEngine, an interactive 3D human generation and editing system that creates 3D digital humans via user-friendly multimodal controls such as natural languages, visual perceptions, and hand-drawing sketche…

Virtual Try-on

FACEMUG: A Multimodal Generative and Fusion Framework for Local Facial Editing

2024-12-26 · Wanglong Lu, Jikai Wang, Xiaogang Jin, Xianta Jiang 외

Existing facial editing methods have achieved remarkable results, yet they often fall short in supporting multimodal conditional local facial editing. One of the significant evidences is that their output image quality d…

AttributeFacial Editing

Multimodal-Conditioned Latent Diffusion Models for Fashion Image Editing

2024-03-21 · Alberto Baldrati, Davide Morelli, Marcella Cornia, Marco Bertini 외

Fashion illustration is a crucial medium for designers to convey their creative vision and transform design concepts into tangible representations that showcase the interplay between clothing and the human body. In the c…

DenoisingVirtual Try-on