Boosting of Head Pose Estimation by Knowledge Distillation

2021-08-20 · Andrey Sheka, Victor Samun

We propose a response-based method of knowledge distillation (KD) for the head pose estimation problem. A student model trained by the proposed KD achieves results better than a teacher model, which is atypical for the response-based method. Our method consists of two stages. In the first stage, we trained the base neural network (NN), which has one regression head and four regression via classification (RvC) heads. We build the convolutional ensemble over the base NN using offsets of face bounding boxes over a regular grid. In the second stage, we perform KD from the convolutional ensemble into the final NN with one RvC head. The KD improves the results by an average of 7.7\% compared to base NN. This feature makes it possible to use KD as a booster and effectively train deeper NNs. NNs trained by our KD method partially improved the state-of-the-art results. KD-ResNet152 has the best results, and KD-ResNet18 has a better result on the AFLW2000 dataset than any previous method.We have made publicly available trained NNs and face bounding boxes for the 300W-LP, AFLW, AFLW2000, and BIWI datasets.Our method potentially can be effective for other regression problems.

📄 PDF Abstract BibTeX arXiv:2108.09183

Code (0)

등록된 구현이 없습니다.

Tasks

Head Pose Estimation Knowledge Distillation Pose Estimation regression

Methods 이 논문이 사용한 방법론

Knowledge Distillation A very simple way to improve the performance of almost any machine learning algorithm is to train many different models on the same data and then to average their predictions.…

ADU-Depth: Attention-based Distillation with Uncertainty Modeling for Depth Estimation

2023-09-26 · Zizhang Wu, Zhuozheng Li, Zhi-Gang Fan, Yunzhe Wu 외

Monocular depth estimation is challenging due to its inherent ambiguity and ill-posed nature, yet it is quite important to many applications. While recent works achieve limited accuracy by designing increasingly complica…

3D geometry Depth Estimation Domain Adaptation Knowledge Distillation+2

Attention is all you need for boosting graph convolutional neural network

2024-03-10 · Yinwei Wu

Graph Convolutional Neural Networks (GCNs) possess strong capabilities for processing graph data in non-grid domains. They can capture the topological logical structure and node features in graphs and integrate them into…

All Knowledge Distillation Recommendation Systems

Fast-HaMeR: Boosting Hand Mesh Reconstruction using Knowledge Distillation

2026-03-17 · Hunain Ahmed Jillani, Ahmed Tawfik Aboukhadra, Ahmed Elhayek, Jameel Malik 외 arxiv

Fast and accurate 3D hand reconstruction is essential for real-time applications in VR/AR, human-computer interaction, robotics, and healthcare. Most state-of-the-art methods rely on heavy models, limiting their use on r…

Knowledge Distillation

BoostingBERT:Integrating Multi-Class Boosting into BERT for NLP Tasks

2020-09-13 · Tongwen Huang, Qingyun She, Junlin Zhang

As a pre-trained Transformer model, BERT (Bidirectional Encoder Representations from Transformers) has achieved ground-breaking performance on multiple NLP tasks. On the other hand, Boosting is a popular ensemble learnin…

Ensemble Learning Knowledge Distillation

Cross-Domain Knowledge Distillation for Low-Resolution Human Pose Estimation

2024-05-19 · Zejun Gu, Zhong-Qiu Zhao, Henghui Ding, Hao Shen 외

In practical applications of human pose estimation, low-resolution inputs frequently occur, and existing state-of-the-art models perform poorly with low-resolution images. This work focuses on boosting the performance of…

Knowledge Distillation Pose Estimation