paper-with-me

홈 › Papers

GeneAvatar: Generic Expression-Aware Volumetric Head Avatar Editing from a Single Image

2024-04-02 · CVPR 2024 1 · Chong Bao, yinda zhang, Yuan Li, Xiyu Zhang, Bangbang Yang, Hujun Bao, Marc Pollefeys, Guofeng Zhang, Zhaopeng Cui

Recently, we have witnessed the explosive growth of various volumetric representations in modeling animatable head avatars. However, due to the diversity of frameworks, there is no practical method to support high-level applications like 3D head avatar editing across different representations. In this paper, we propose a generic avatar editing approach that can be universally applied to various 3DMM driving volumetric head avatars. To achieve this goal, we design a novel expression-aware modification generative model, which enables lift 2D editing from a single image to a consistent 3D modification field. To ensure the effectiveness of the generative modification process, we develop several techniques, including an expression-dependent modification distillation scheme to draw knowledge from the large-scale head avatar model and 2D facial texture editing tools, implicit latent space guidance to enhance model convergence, and a segmentation-based loss reweight strategy for fine-grained texture inversion. Extensive experiments demonstrate that our method delivers high-quality and consistent results across multiple expression and viewpoints. Project page: https://zju3dv.github.io/geneavatar/

📄 PDF Abstract BibTeX arXiv:2404.02152

Code (0)

등록된 구현이 없습니다.

Tasks

Diversity

Similar Papers 제목 키워드 기반

VOODOO 3D: Volumetric Portrait Disentanglement for One-Shot 3D Head Reenactment

2023-12-07 · CVPR 2024 1 · Phong Tran, Egor Zakharov, Long-Nhat Ho, Anh Tuan Tran 외

We present a 3D-aware one-shot head reenactment method based on a fully volumetric neural disentanglement framework for source appearance and driver expressions. Our method is real-time and produces high-fidelity and vie…

DisentanglementSelf-Supervised Learning

Headset: Human emotion awareness under partial occlusions multimodal dataset

2024-02-14 · IEEE Transactions on Visualization and Computer Graphics- ISMAR 2023 10 · Fatemeh Ghorbani Lohesara, Davi Rabbouni Freitas, Christine Guillemot, Karen Eguiazarian 외

The volumetric representation of human interactions is one of the fundamental domains in the development of immersive media productions and telecommunication applications. Particularly in the context of the rapid advance…

Facial Expression RecognitionPoint cloud reconstruction

OmniAvatar: Geometry-Guided Controllable 3D Head Synthesis

2023-03-27 · CVPR 2023 1 · Hongyi Xu, Guoxian Song, Zihang Jiang, Jianfeng Zhang 외

We present OmniAvatar, a novel geometry-guided 3D head synthesis model trained from in-the-wild unstructured images that is capable of synthesizing diverse identity-preserved 3D heads with compelling dynamic details unde…

DPHMs: Diffusion Parametric Head Models for Depth-based Tracking

2023-12-02 · CVPR 2024 1 · Jiapeng Tang, Angela Dai, Yinyu Nie, Lev Markhasin 외

We introduce Diffusion Parametric Head Models (DPHMs), a generative model that enables robust volumetric head reconstruction and tracking from monocular depth sequences. While recent volumetric head models, such as NPHMs…

Learning Personalized High Quality Volumetric Head Avatars from Monocular RGB Videos

2023-04-04 · CVPR 2023 1 · Ziqian Bai, Feitong Tan, Zeng Huang, Kripasindhu Sarkar 외

We propose a method to learn a high-quality implicit 3D head avatar from a monocular RGB video captured in the wild. The learnt avatar is driven by a parametric face model to achieve user-controlled facial expressions an…

Face ModelVocal Bursts Intensity Prediction