MarginDistillation: distillation for margin-based softmax
The usage of convolutional neural networks (CNNs) in conjunction with a margin-based softmax approach demonstrates a state-of-the-art performance for the face recognition problem. Recently, lightweight neural network models trained with the margin-based softmax have been introduced for the face identification task for edge devices. In this paper, we propose a novel distillation method for lightweight neural network architectures that outperforms other known methods for the face recognition task on LFW, AgeDB-30 and Megaface datasets. The idea of the proposed method is to use class centers from the teacher network for the student network. Then the student network is trained to get the same angles between the class centers and the face embeddings, predicted by the teacher network.
Code (1)
Tasks
Face IdentificationFace RecognitionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Real Additive Margin Softmax for Speaker Verification
The additive margin softmax (AM-Softmax) loss has delivered remarkable performance in speaker verification. A supposed behavior of AM-Softmax is that it can shrink within-class variation by putting emphasis on target log…
Speaker VerificationAdditive Margin Softmax for Face Verification
In this paper, we propose a conceptually simple and geometrically interpretable objective function, i.e. additive margin Softmax (AM-Softmax), for deep face verification. In general, the face verification task can be vie…
Face VerificationMetric LearningAdaDistill: Adaptive Knowledge Distillation for Deep Face Recognition
Knowledge distillation (KD) aims at improving the performance of a compact student model by distilling the knowledge from a high-performing teacher model. In this paper, we present an adaptive KD approach, namely AdaDist…
Face RecognitionKnowledge DistillationInvestigation of Large-Margin Softmax in Neural Language Modeling
To encourage intra-class compactness and inter-class separability among trainable feature vectors, large-margin softmax methods are developed and widely applied in the face recognition community. The introduction of the …
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Face RecognitionLanguage Modeling+3ProxylessKD: Direct Knowledge Distillation with Inherited Classifier for Face Recognition
Knowledge Distillation (KD) refers to transferring knowledge from a large model to a smaller one, which is widely used to enhance model performance in machine learning. It tries to align embedding spaces generated from t…
Face RecognitionKnowledge Distillation