홈 › Papers

FaceFolds: Meshed Radiance Manifolds for Efficient Volumetric Rendering of Dynamic Faces

2024-04-22 · Safa C. Medin, Gengyan Li, Ruofei Du, Stephan Garbin, Philip Davidson, Gregory W. Wornell, Thabo Beeler, Abhimitra Meka

3D rendering of dynamic face captures is a challenging problem, and it demands improvements on several fronts$\unicode{x2014}$photorealism, efficiency, compatibility, and configurability. We present a novel representation that enables high-quality volumetric rendering of an actor's dynamic facial performances with minimal compute and memory footprint. It runs natively on commodity graphics soft- and hardware, and allows for a graceful trade-off between quality and efficiency. Our method utilizes recent advances in neural rendering, particularly learning discrete radiance manifolds to sparsely sample the scene to model volumetric effects. We achieve efficient modeling by learning a single set of manifolds for the entire dynamic sequence, while implicitly modeling appearance changes as temporal canonical texture. We export a single layered mesh and view-independent RGBA texture video that is compatible with legacy graphics renderers without additional ML integration. We demonstrate our method by rendering dynamic face captures of real actors in a game engine, at comparable photorealism to state-of-the-art neural rendering techniques at previously unseen frame rates.

📄 PDF Abstract BibTeX arXiv:2404.13807

Code (0)

등록된 구현이 없습니다.

Tasks

Neural Rendering

Methods 이 논문이 사용한 방법론

SET Dynamic Sparse Training method where weight mask is updated randomly periodically

LegacyAvatars: Volumetric Face Avatars For Traditional Graphics Pipelines

2026-01-18 · Safa C. Medin, Gengyan Li, Ziqian Bai, Ruofei Du 외 arxiv

We introduce a novel representation for efficient classical rendering of photorealistic 3D face avatars. Leveraging recent advances in radiance fields anchored to parametric face models, our approach achieves controllabl…

GRAM: Generative Radiance Manifolds for 3D-Aware Image Generation

2021-12-16 · CVPR 2022 1 · Yu Deng, Jiaolong Yang, Jianfeng Xiang, Xin Tong

3D-aware image generative modeling aims to generate 3D-consistent images with explicitly controllable camera poses. Recent works have shown promising results by training neural radiance field (NeRF) generators on unstruc…

3D-Aware Image Synthesis Face Generation Image Generation NeRF+1

Flash Cache: Reducing Bias in Radiance Cache Based Inverse Rendering

2024-09-09 · Benjamin Attal, Dor Verbin, Ben Mildenhall, Peter Hedman 외

State-of-the-art techniques for 3D reconstruction are largely based on volumetric scene representations, which require sampling multiple points to compute the color arriving along a ray. Using these representations for m…

3D Reconstruction Inverse Rendering

MERF: Memory-Efficient Radiance Fields for Real-time View Synthesis in Unbounded Scenes

2023-02-23 · Christian Reiser, Richard Szeliski, Dor Verbin, Pratul P. Srinivasan 외

Neural radiance fields enable state-of-the-art photorealistic view synthesis. However, existing radiance field representations are either too compute-intensive for real-time rendering or require too much memory to scale …

Adaptive Shells for Efficient Neural Radiance Field Rendering

2023-11-16 · Zian Wang, Tianchang Shen, Merlin Nimier-David, Nicholas Sharp 외

Neural radiance fields achieve unprecedented quality for novel view synthesis, but their volumetric formulation remains expensive, requiring a huge number of samples to render high-resolution images. Volumetric encodings…

Novel View Synthesis Stochastic Optimization