Blended Point Cloud Diffusion for Localized Text-guided Shape Editing
Natural language offers a highly intuitive interface for enabling localized fine-grained edits of 3D shapes. However, prior works face challenges in preserving global coherence while locally modifying the input 3D shape. In this work, we introduce an inpainting-based framework for editing shapes represented as point clouds. Our approach leverages foundation 3D diffusion models for achieving localized shape edits, adding structural guidance in the form of a partial conditional shape, ensuring that other regions correctly preserve the shape's identity. Furthermore, to encourage identity preservation also within the local edited region, we propose an inference-time coordinate blending algorithm which balances reconstruction of the full shape with inpainting at a progression of noise levels during the inference process. Our coordinate blending algorithm seamlessly blends the original shape with its edited version, enabling a fine-grained editing of 3D shapes, all while circumventing the need for computationally expensive and often inaccurate inversion. Extensive experiments show that our method outperforms alternative techniques across a wide range of metrics that evaluate both fidelity to the original shape and also adherence to the textual description.
Code (0)
등록된 구현이 없습니다.
Tasks
Point CloudsSimilar Papers 제목 키워드 기반
BlendedNet: A Blended Wing Body Aircraft Dataset and Surrogate Model for Aerodynamic Predictions
BlendedNet is a publicly available aerodynamic dataset of 999 blended wing body (BWB) geometries. Each geometry is simulated across about nine flight conditions, yielding 8830 converged RANS cases with the Spalart-Allmar…
Point CloudsBlended Convolution and Synthesis for Efficient Discrimination of 3D Shapes
Existing networks directly learn feature representations on 3D point clouds for shape analysis. We argue that 3D point clouds are highly redundant and hold irregular (permutation-invariant) structure, which makes it diff…
3D Shape RecognitionRetrievalTranslationMeltdown: Circuits and Bifurcations in Point-Cloud-Conditioned 3D Diffusion Transformers
Sparse point clouds are a common input modality for 3D surface reconstruction, including in safety-critical settings such as surgical navigation and autonomous perception. Recent point-cloud-conditioned 3D diffusion tran…
Point CloudsHigh-Resolution Image Editing via Multi-Stage Blended Diffusion
Diffusion models have shown great results in image generation and in image editing. However, current approaches are limited to low resolutions due to the computational cost of training diffusion models for high-resolutio…
Image GenerationImage InpaintingSuper-ResolutionVocal Bursts Intensity PredictionHarnessing Text-to-Image Diffusion Models for Point Cloud Self-Supervised Learning
Diffusion-based models, widely used in text-to-image generation, have proven effective in 2D representation learning. Recently, this framework has been extended to 3D self-supervised learning by constructing a conditiona…
Self-Supervised LearningText-to-Image GenerationRepresentation LearningPoint Clouds