Neural Point Cloud Diffusion for Disentangled 3D Shape and Appearance Generation

2023-12-21 · CVPR 2024 1 · Philipp Schröppel, Christopher Wewer, Jan Eric Lenssen, Eddy Ilg, Thomas Brox

Controllable generation of 3D assets is important for many practical applications like content creation in movies, games and engineering, as well as in AR/VR. Recently, diffusion models have shown remarkable results in generation quality of 3D objects. However, none of the existing models enable disentangled generation to control the shape and appearance separately. For the first time, we present a suitable representation for 3D diffusion models to enable such disentanglement by introducing a hybrid point cloud and neural radiance field approach. We model a diffusion process over point positions jointly with a high-dimensional feature space for a local density and radiance decoder. While the point positions represent the coarse shape of the object, the point features allow modeling the geometry and appearance details. This disentanglement enables us to sample both independently and therefore to control both separately. Our approach sets a new state of the art in generation compared to previous disentanglement-capable methods by reduced FID scores of 30-90% and is on-par with other non disentanglement-capable state-of-the art methods.

📄 PDF Abstract BibTeX arXiv:2312.14124

Code (1)

lmb-freiburg/neural-point-cloud-diffusion 공식 구현 pytorch

Tasks

Decoder Disentanglement

Methods 이 논문이 사용한 방법론

Diffusion Diffusion models generate samples by gradually removing noise from a signal, and their training objective can be expressed as a reweighted variational lower-bound…

NON 설명 없음

Sketch and Text Guided Diffusion Model for Colored Point Cloud Generation

2023-08-05 · ICCV 2023 1 · Zijie Wu, Yaonan Wang, Mingtao Feng, He Xie 외

Diffusion probabilistic models have achieved remarkable success in text guided image generation. However, generating 3D shapes is still challenging due to the lack of sufficient data containing 3D models along with their…

Denoising Image Generation Point Cloud Generation

EditVAE: Unsupervised Part-Aware Controllable 3D Point Cloud Shape Generation

2021-10-13 · Shidi Li, Miaomiao Liu, Christian Walder

This paper tackles the problem of parts-aware point cloud generation. Unlike existing works which require the point cloud to be segmented into parts a priori, our parts-aware editing and generation are performed in an un…

Inductive Bias Point Cloud Generation

Few-shot point cloud reconstruction and denoising via learned Guassian splats renderings and fine-tuned diffusion features

2024-04-01 · Pietro Bonazzi, Marie-Julie Rakatosaona, Marco Cannici, Federico Tombari 외

Existing deep learning methods for the reconstruction and denoising of point clouds rely on small datasets of 3D shapes. We circumvent the problem by leveraging deep learning methods trained on billions of images. We pro…

3D Reconstruction Deep Learning Denoising Point cloud reconstruction

Points-to-3D: Bridging the Gap between Sparse Points and Shape-Controllable Text-to-3D Generation

2023-07-26 · Chaohui Yu, Qiang Zhou, Jingliang Li, Zhe Zhang 외

Text-to-3D generation has recently garnered significant attention, fueled by 2D diffusion models trained on billions of image-text pairs. Existing methods primarily rely on score distillation to leverage the 2D diffusion…

3D Generation NeRF Text to 3D

MRGAN: Multi-Rooted 3D Shape Generation with Unsupervised Part Disentanglement

2020-07-25 · Rinon Gal, Amit Bermano, Hao Zhang, Daniel Cohen-Or

We present MRGAN, a multi-rooted adversarial network which generates part-disentangled 3D point-cloud shapes without part-based shape supervision. The network fuses multiple branches of tree-structured graph convolution …

3D Shape Generation Disentanglement