Exclusivity-Consistency Regularized Knowledge Distillation for Face Recognition
Knowledge distillation is an effective tool to compress large pre-trained Convolutional Neural Networks (CNNs) or their ensembles into models applicable to mobile and embedded devices. The success of which mainly comes from two aspects: the designed student network and the exploited knowledge. However, current methods usually suffer from the low-capability of mobile-level student network and the unsatisfactory knowledge for distillation. In this paper, we propose a novel position-aware exclusivity to encourage large diversity among different filters of the same layer to alleviate the low-capability of student network. Moreover, we investigate the effect of several prevailing knowledge for face recognition distillation and conclude that the knowledge of feature consistency is more flexible and preserves much more information than others. Experiments on a variety of face recognition benchmarks have revealed the superiority of our method over the state-of-the-arts.
Code (0)
등록된 구현이 없습니다.
Tasks
DiversityFace RecognitionKnowledge DistillationPositionSimilar Papers 제목 키워드 기반
Exclusivity Regularized Machine
It has been recognized that the diversity of base learners is of utmost importance to a good ensemble. This paper defines a novel measurement of diversity, termed as exclusivity. With the designed exclusivity, we further…
DiversityExclusivity-Consistency Regularized Multi-View Subspace Clustering
Multi-view subspace clustering aims to partition a set of multi-source data into their underlying groups. To boost the performance of multi-view clustering, numerous subspace learning algorithms have been developed in re…
ClusteringMulti-view Subspace ClusteringPositionICD-Face: Intra-class Compactness Distillation for Face Recognition
Knowledge distillation is an effective model compression method to improve the performance of a lightweight student model by transferring the knowledge of a well-performed teacher model, which has been widely adopted…
Face RecognitionKnowledge DistillationModel CompressionCoupleFace: Relation Matters for Face Recognition Distillation
Knowledge distillation is an effective method to improve the performance of a lightweight neural network (i.e., student model) by transferring the knowledge of a well-performed neural network (i.e., teacher model), which…
Face RecognitionKnowledge DistillationRelationPeriocular Embedding Learning with Consistent Knowledge Distillation from Face
Periocular biometric, the peripheral area of the ocular, is a collaborative alternative to the face, especially when the face is occluded or masked. However, in practice, sole periocular biometric capture the least salie…
Knowledge DistillationPrediction