paper-with-me

Papers

An Implicit Physical Face Model Driven by Expression and Style

2024-01-27 · Lingchen Yang, Gaspard Zoss, Prashanth Chandran, Paulo Gotardo, Markus Gross, Barbara Solenthaler, Eftychios Sifakis, Derek Bradley

3D facial animation is often produced by manipulating facial deformation models (or rigs), that are traditionally parameterized by expression controls. A key component that is usually overlooked is expression 'style', as in, how a particular expression is performed. Although it is common to define a semantic basis of expressions that characters can perform, most characters perform each expression in their own style. To date, style is usually entangled with the expression, and it is not possible to transfer the style of one character to another when considering facial animation. We present a new face model, based on a data-driven implicit neural physics model, that can be driven by both expression and style separately. At the core, we present a framework for learning implicit physics-based actuations for multiple subjects simultaneously, trained on a few arbitrary performance capture sequences from a small set of identities. Once trained, our method allows generalized physics-based facial animation for any of the trained identities, extending to unseen performances. Furthermore, it grants control over the animation style, enabling style transfer from one character to another or blending styles of different characters. Lastly, as a physics-based model, it is capable of synthesizing physical effects, such as collision handling, setting our method apart from conventional approaches.

📄 PDF Abstract BibTeX arXiv:2401.15414

Code (0)

등록된 구현이 없습니다.

Tasks

Face ModelStyle Transfer

Methods 이 논문이 사용한 방법론

SET Dynamic Sparse Training method where weight mask is updated randomly periodically

Similar Papers 제목 키워드 기반

Parametric Implicit Face Representation for Audio-Driven Facial Reenactment

2023-06-13 · CVPR 2023 1 · Ricong Huang, Peiwen Lai, Yipeng Qin, Guanbin Li

Audio-driven facial reenactment is a crucial technique that has a range of applications in film-making, virtual avatars and video conferences. Existing works either employ explicit intermediate face representations (e.g.…

Data AugmentationImage GenerationImage Inpainting

i3DMM: Deep Implicit 3D Morphable Model of Human Heads

2020-11-28 · CVPR 2021 1 · Tarun Yenamandra, Ayush Tewari, Florian Bernard, Hans-Peter Seidel 외

We present the first deep implicit 3D morphable model (i3DMM) of full heads. Unlike earlier morphable face models it not only captures identity-specific geometry, texture, and expressions of the frontal face, but also mo…

Imitator: Personalized Speech-driven 3D Facial Animation

2022-12-30 · ICCV 2023 1 · Balamurugan Thambiraja, Ikhsanul Habibie, Sadegh Aliakbarian, Darren Cosker 외

Speech-driven 3D facial animation has been widely explored, with applications in gaming, character animation, virtual reality, and telepresence systems. State-of-the-art methods deform the face topology of the target act…

PC-Talk: Precise Facial Animation Control for Audio-Driven Talking Face Generation

2025-03-18 · Baiqin Wang, Xiangyu Zhu, Fan Shen, Hao Xu 외

Recent advancements in audio-driven talking face generation have made great progress in lip synchronization. However, current methods often lack sufficient control over facial animation such as speaking style and emotion…

Face GenerationTalking Face Generation

SadTalker: Learning Realistic 3D Motion Coefficients for Stylized Audio-Driven Single Image Talking Face Animation

2022-11-22 · CVPR 2023 1 · Wenxuan Zhang, Xiaodong Cun, Xuan Wang, Yong Zhang 외

Generating talking head videos through a face image and a piece of speech audio still contains many challenges. ie, unnatural head movement, distorted expression, and identity modification. We argue that these issues are…

Image AnimationTalking Head Generation