Task Decoupled Knowledge Distillation For Lightweight Face Detectors
We propose a knowledge distillation method for the face detection task. This method decouples the distillation task of face detection into two subtasks, i.e., the classification distillation subtask and the regression distillation subtask. We add the task-specific convolutions in the teacher network and add the adaption convolutions on the feature maps of the student network to generate the task decoupled features. Then, each subtask uses different samples in distilling the features to be consistent with the corresponding detection subtask. Moreover, we propose an effective probability distillation method to joint boost the accuracy of the student network.
Code (1)
Tasks
Face DetectionKnowledge DistillationregressionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Adaptive Decoupled Pose Knowledge Distillation
Existing state-of-the-art human pose estimation approaches require heavy computational resources for accurate prediction. One promising technique to obtain an accurate yet lightweight pose estimator is Knowledge Distilla…
Knowledge DistillationPose EstimationDecoupled Smart Contract Audits: Lightweight LLM Framework via Distillation and Aggregation
Smart contracts face critical security challenges that require thorough auditing in decentralized web services. While Large Language Models (LLMs) have shown promise in automated vulnerability detection, existing approac…
Vulnerability DetectionKnowledge DistillationRemote Sensing Image Classification with Decoupled Knowledge Distillation
To address the challenges posed by the large number of parameters in existing remote sensing image classification models, which hinder deployment on resource-constrained devices, this paper proposes a lightweight classif…
Classificationimage-classificationImage ClassificationKnowledge Distillation+1BearingPGA-Net: A Lightweight and Deployable Bearing Fault Diagnosis Network via Decoupled Knowledge Distillation and FPGA Acceleration
Deep learning has achieved remarkable success in the field of bearing fault diagnosis. However, this success comes with larger models and more complex computations, which cannot be transferred into industrial fields requ…
CPUFault DiagnosisKnowledge DistillationQuantizationScale Decoupled Distillation
Logit knowledge distillation attracts increasing attention due to its practicality in recent studies. However, it often suffers inferior performance compared to the feature knowledge distillation. In this paper, we argue…
Knowledge Distillation