Dynamic Facial Analysis: From Bayesian Filtering to Recurrent Neural Network
Facial analysis in videos, including head pose estimation and facial landmark localization, is key for many applications such as facial animation capture, human activity recognition, and human-computer interaction. In this paper, we propose to use a recurrent neural network (RNN) for joint estimation and tracking of facial features in videos. We are inspired by the fact that the computation performed in an RNN bears resemblance to Bayesian filters, which have been used for tracking in many previous methods for facial analysis from videos. Bayesian filters used in these methods, however, require complicated, problem-specific design and tuning. In contrast, our proposed RNN-based method avoids such tracker-engineering by learning from training data, similar to how a convolutional neural network (CNN) avoids feature-engineering for image classification. As an end-to-end network, the proposed RNN-based method provides a generic and holistic solution for joint estimation and tracking of various types of facial features from consecutive video frames. Extensive experimental results on head pose estimation and facial landmark localization from videos demonstrate that the proposed RNN-based method outperforms frame-wise models and Bayesian filtering. In addition, we create a large-scale synthetic dataset for head pose estimation, with which we achieve state-of-the-art performance on a benchmark dataset.
Code (0)
등록된 구현이 없습니다.
Tasks
Activity RecognitionFace AlignmentFeature EngineeringHead Pose EstimationHuman Activity Recognitionimage-classificationImage ClassificationPose EstimationSimilar Papers 제목 키워드 기반
Recurrent Neural Filters: Learning Independent Bayesian Filtering Steps for Time Series Prediction
Despite the recent popularity of deep generative state space models, few comparisons have been made between network architectures and the inference steps of the Bayesian filtering framework -- with most models simultaneo…
State Space ModelsTime SeriesTime Series AnalysisTime Series PredictionFunctional Bayesian Filter
We present a general nonlinear Bayesian filter for high-dimensional state estimation using the theory of reproducing kernel Hilbert space (RKHS). Applying kernel method and the representer theorem to perform linear quadr…
State EstimationTime SeriesTime Series AnalysisCoupled Variational Recurrent Collaborative Filtering
We focus on the problem of streaming recommender system and explore novel collaborative filtering algorithms to handle the data dynamicity and complexity in a streaming manner. Although deep neural networks have demonstr…
Collaborative FilteringRecommendation SystemsVariational InferenceExplainable Gated Bayesian Recurrent Neural Network for Non-Markov State Estimation
The optimality of Bayesian filtering relies on the completeness of prior models, while deep learning holds a distinct advantage in learning models from offline data. Nevertheless, the current fusion of these two methodol…
Computational EfficiencyState EstimationThe Neural Particle Filter
The robust estimation of dynamically changing features, such as the position of prey, is one of the hallmarks of perception. On an abstract, algorithmic level, nonlinear Bayesian filtering, i.e. the estimation of tempora…
State Estimation