Fast Bi-layer Neural Synthesis of One-Shot Realistic Head Avatars
We propose a neural rendering-based system that creates head avatars from a single photograph. Our approach models a person's appearance by decomposing it into two layers. The first layer is a pose-dependent coarse image that is synthesized by a small neural network. The second layer is defined by a pose-independent texture image that contains high-frequency details. The texture image is generated offline, warped and added to the coarse image to ensure a high effective resolution of synthesized head views. We compare our system to analogous state-of-the-art systems in terms of visual quality and speed. The experiments show significant inference speedup over previous neural head avatar models for a given visual quality. We also report on a real-time smartphone-based implementation of our system.
Code (1)
Tasks
Neural RenderingTalking Head GenerationSimilar Papers 제목 키워드 기반
AE-NeRF: Audio Enhanced Neural Radiance Field for Few Shot Talking Head Synthesis
Audio-driven talking head synthesis is a promising topic with wide applications in digital human, film making and virtual reality. Recent NeRF-based approaches have shown superiority in quality and fidelity compared to p…
Face GenerationNeRFTalking Head GenerationFrameNeRF: A Simple and Efficient Framework for Few-shot Novel View Synthesis
We present a novel framework, called FrameNeRF, designed to apply off-the-shelf fast high-fidelity NeRF models with fast training speed and high rendering quality for few-shot novel view synthesis tasks. The training sta…
NeRFNovel View SynthesisBakedAvatar: Baking Neural Fields for Real-Time Head Avatar Synthesis
Synthesizing photorealistic 4D human head avatars from videos is essential for VR/AR, telepresence, and video game applications. Although existing Neural Radiance Fields (NeRF)-based methods achieve high-fidelity results…
Face ReenactmentNeRFMask-off: Synthesizing Face Images in the Presence of Head-mounted Displays
A head-mounted display (HMD) could be an important component of augmented reality system. However, as the upper face region is seriously occluded by the device, the user experience could be affected in applications such …
ColorizationFace AlignmentFace GenerationFast Spectrogram Inversion using Multi-head Convolutional Neural Networks
We propose the multi-head convolutional neural network (MCNN) architecture for waveform synthesis from spectrograms. Nonlinear interpolation in MCNN is employed with transposed convolution layers in parallel heads. MCNN …
speech-recognitionSpeech RecognitionSpeech Synthesis