IPVTON: Image-based 3D Virtual Try-on with Image Prompt Adapter
Given a pair of images depicting a person and a garment separately, image-based 3D virtual try-on methods aim to reconstruct a 3D human model that realistically portrays the person wearing the desired garment. In this paper, we present IPVTON, a novel image-based 3D virtual try-on framework. IPVTON employs score distillation sampling with image prompts to optimize a hybrid 3D human representation, integrating target garment features into diffusion priors through an image prompt adapter. To avoid interference with non-target areas, we leverage mask-guided image prompt embeddings to focus the image features on the try-on regions. Moreover, we impose geometric constraints on the 3D model with a pseudo silhouette generated by ControlNet, ensuring that the clothed 3D human model retains the shape of the source identity while accurately wearing the target garments. Extensive qualitative and quantitative experiments demonstrate that IPVTON outperforms previous methods in image-based 3D virtual try-on tasks, excelling in both geometry and texture.
Code (0)
등록된 구현이 없습니다.
Tasks
Virtual Try-onMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Unpaired Multi-Domain Histopathology Virtual Staining using Dual Path Prompted Inversion
Virtual staining leverages computer-aided techniques to transfer the style of histochemically stained tissue samples to other staining types. In virtual staining of pathological images, maintaining strict structural cons…
DiagnosticDisentanglementPrompt LearningStyle Transfer+1IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models
Recent years have witnessed the strong power of large text-to-image diffusion models for the impressive generative capability to create high-fidelity images. However, it is very tricky to generate desired images using on…
Diffusion Personalization Tuning FreeImage GenerationPersonalized Image GenerationPrompt EngineeringIPAdapter-Instruct: Resolving Ambiguity in Image-based Conditioning using Instruct Prompts
Diffusion models continuously push the boundary of state-of-the-art image generation, but the process is hard to control with any nuance: practice proves that textual prompts are inadequate for accurately describing imag…
Image GenerationStyle TransferSUR-adapter: Enhancing Text-to-Image Pre-trained Diffusion Models with Large Language Models
Diffusion models, which have emerged to become popular text-to-image generation models, can produce high-quality and content-rich images guided by textual prompts. However, there are limitations to semantic understanding…
Image GenerationKnowledge Distillationparameter-efficient fine-tuningText to Image Generation+1PA-SAM: Prompt Adapter SAM for High-Quality Image Segmentation
The Segment Anything Model (SAM) has exhibited outstanding performance in various image segmentation tasks. Despite being trained with over a billion masks, SAM faces challenges in mask prediction quality in numerous sce…
DecoderImage SegmentationSegmentationSemantic Segmentation