GaussianAnything: Interactive Point Cloud Flow Matching For 3D Object Generation

2024-11-12 · Yushi Lan, Shangchen Zhou, Zhaoyang Lyu, Fangzhou Hong, Shuai Yang, Bo Dai, Xingang Pan, Chen Change Loy

While 3D content generation has advanced significantly, existing methods still face challenges with input formats, latent space design, and output representations. This paper introduces a novel 3D generation framework that addresses these challenges, offering scalable, high-quality 3D generation with an interactive Point Cloud-structured Latent space. Our framework employs a Variational Autoencoder (VAE) with multi-view posed RGB-D(epth)-N(ormal) renderings as input, using a unique latent space design that preserves 3D shape information, and incorporates a cascaded latent flow-based model for improved shape-texture disentanglement. The proposed method, GaussianAnything, supports multi-modal conditional 3D generation, allowing for point cloud, caption, and single image inputs. Notably, the newly proposed latent space naturally enables geometry-texture disentanglement, thus allowing 3D-aware editing. Experimental results demonstrate the effectiveness of our approach on multiple datasets, outperforming existing native 3D methods in both text- and image-conditioned 3D generation.

📄 PDF Abstract BibTeX arXiv:2411.08033

Code (0)

등록된 구현이 없습니다.

Tasks

3D Generation Disentanglement

Methods 이 논문이 사용한 방법론

Diffusion Diffusion models generate samples by gradually removing noise from a signal, and their training objective can be expressed as a reweighted variational lower-bound…

Latent Diffusion Model Diffusion models applied to latent spaces, which are normally built with (Variational) Autoencoders.

Efficient Point Clouds Upsampling via Flow Matching

2025-01-25 · Zhi-Song Liu, Chenhang He, Lei LI

Diffusion models are a powerful framework for tackling ill-posed problems, with recent advancements extending their use to point cloud upsampling. Despite their potential, existing diffusion models struggle with ineffici…

point cloud upsampling

SCTN: Sparse Convolution-Transformer Network for Scene Flow Estimation

2021-05-10 · Bing Li, Cheng Zheng, Silvio Giancola, Bernard Ghanem

We propose a novel scene flow estimation approach to capture and infer 3D motions from point clouds. Estimating 3D motions for point clouds is challenging, since a point cloud is unordered and its density is significantl…

Scene Flow Estimation

Self-Point-Flow: Self-Supervised Scene Flow Estimation from Point Clouds with Optimal Transport and Random Walk

2021-05-18 · CVPR 2021 1 · Ruibo Li, Guosheng Lin, Lihua Xie

Due to the scarcity of annotated scene flow data, self-supervised scene flow learning in point clouds has attracted increasing attention. In the self-supervised manner, establishing correspondences between two point clou…

Scene Flow Estimation Self-Supervised Learning Self-supervised Scene Flow Estimation

PUFM++: Point Cloud Upsampling via Enhanced Flow Matching

2025-12-24 · Zhi-Song Liu, Chenhang He, Roland Maier, Andreas Rupp arxiv

Recent advances in generative modeling have demonstrated strong promise for high-quality point cloud upsampling. In this work, we present PUFM++, an enhanced flow-matching framework for reconstructing dense and accurate …

Point Clouds

GMSF: Global Matching Scene Flow

2023-05-27 · NeurIPS 2023 11 · Yushan Zhang, Johan Edstedt, Bastian Wandt, Per-Erik Forssén 외

We tackle the task of scene flow estimation from point clouds. Given a source and a target point cloud, the objective is to estimate a translation from each point in the source point cloud to the target, resulting in a 3…