Dynamic 3D Avatar Creation from Hand-held Video Input
We present a complete pipeline for creating fully rigged, personalized 3D facial avatars from hand-held video. Our system faithfully recovers facial expression dynamics of the user by adapting a blendshape template to an image sequence of recorded expressions using an optimization that integrates feature tracking, optical flow, and shape from shading. Fine-scale details such as wrinkles are captured separately in normal maps and ambient occlusion maps. From this user- and expression-specific data, we learn a regressor for on-the-fly detail synthesis during animation to enhance the perceptual realism of the avatars. Our system demonstrates that the use of appropriate reconstruction priors yields compelling face rigs even with a minimalistic acquisition system and limited user assistance. This facilitates a range of new applications in computer animation and consumer-level online communication based on personalized avatars. We present realtime application demos to validate our method
Code (0)
등록된 구현이 없습니다.
Tasks
Optical Flow EstimationSimilar Papers 제목 키워드 기반
HARP: Personalized Hand Reconstruction from a Monocular RGB Video
We present HARP (HAnd Reconstruction and Personalization), a personalized hand avatar creation approach that takes a short monocular RGB video of a human hand as input and reconstructs a faithful hand avatar exhibiting a…
3D Hand Pose EstimationHand Pose EstimationPose EstimationWildAvatar: Learning In-the-wild 3D Avatars from the Web
Existing research on avatar creation is typically limited to laboratory datasets, which require high costs against scalability and exhibit insufficient representation of the real world. On the other hand, the web abo…
WildAvatar: Web-scale In-the-wild Video Dataset for 3D Avatar Creation
Existing human datasets for avatar creation are typically limited to laboratory environments, wherein high-quality annotations (e.g., SMPL estimation from 3D scans or multi-view images) can be ideally provided. However, …
One2Avatar: Generative Implicit Head Avatar For Few-shot User Adaptation
Traditional methods for constructing high-quality, personalized head avatars from monocular videos demand extensive face captures and training time, posing a significant challenge for scalability. This paper introduces a…
Camera CalibrationMPMAvatar: Learning 3D Gaussian Avatars with Accurate and Robust Physics-Based Dynamics
While there has been significant progress in the field of 3D avatar creation from visual observations, modeling physically plausible dynamics of humans with loose garments remains a challenging problem. Although a few ex…