paper-with-me

Papers

Point-Based Radiance Fields for Controllable Human Motion Synthesis

2023-10-05 · HaiTao Yu, Deheng Zhang, Peiyuan Xie, Tianyi Zhang

This paper proposes a novel controllable human motion synthesis method for fine-level deformation based on static point-based radiance fields. Although previous editable neural radiance field methods can generate impressive results on novel-view synthesis and allow naive deformation, few algorithms can achieve complex 3D human editing such as forward kinematics. Our method exploits the explicit point cloud to train the static 3D scene and apply the deformation by encoding the point cloud translation using a deformation MLP. To make sure the rendering result is consistent with the canonical space training, we estimate the local rotation using SVD and interpolate the per-point rotation to the query view direction of the pre-trained radiance field. Extensive experiments show that our approach can significantly outperform the state-of-the-art on fine-level complex deformation which can be generalized to other 3D characters besides humans.

📄 PDF Abstract BibTeX arXiv:2310.03375

Code (1)

dehezhang2/point_based_nerf_editing 공식 구현 pytorch

Tasks

Motion SynthesisNovel View Synthesis

Similar Papers 제목 키워드 기반

VeRi3D: Generative Vertex-based Radiance Fields for 3D Controllable Human Image Synthesis

2023-09-09 · ICCV 2023 1 · Xinya Chen, Jiaxin Huang, Yanrui Bin, Lu Yu 외

Unsupervised learning of 3D-aware generative adversarial networks has lately made much progress. Some recent work demonstrates promising results of learning human generative models using neural articulated radiance field…

Image Generation

NeRSemble: Multi-view Radiance Field Reconstruction of Human Heads

2023-05-04 · Tobias Kirschstein, Shenhan Qian, Simon Giebenhain, Tim Walter 외

We focus on reconstructing high-fidelity radiance fields of human heads, capturing their animations over time, and synthesizing re-renderings from novel viewpoints at arbitrary time steps. To this end, we propose a new m…

Controllable Radiance Fields for Dynamic Face Synthesis

2022-10-11 · Peiye Zhuang, Liqian Ma, Oluwasanmi Koyejo, Alexander G. Schwing

Recent work on 3D-aware image synthesis has achieved compelling results using advances in neural rendering. However, 3D-aware synthesis of face dynamics hasn't received much attention. Here, we study how to explicitly co…

3D-Aware Image SynthesisFace GenerationFace ParsingImage Generation+1

Motion-Oriented Compositional Neural Radiance Fields for Monocular Dynamic Human Modeling

2024-07-16 · Jaehyeok Kim, Dongyoon Wee, Dan Xu

This paper introduces Motion-oriented Compositional Neural Radiance Fields (MoCo-NeRF), a framework designed to perform free-viewpoint rendering of monocular human videos via novel non-rigid motion modeling approach. In …

NeRF

Surface-Aligned Neural Radiance Fields for Controllable 3D Human Synthesis

2022-01-05 · CVPR 2022 1 · Tianhan Xu, Yasuhiro Fujita, Eiichi Matsumoto

We propose a new method for reconstructing controllable implicit 3D human models from sparse multi-view RGB videos. Our method defines the neural scene representation on the mesh surface points and signed distances from …

NeRF