Leveraging ASR Pretrained Conformers for Speaker Verification through Transfer Learning and Knowledge Distillation
This paper explores the use of ASR-pretrained Conformers for speaker verification, leveraging their strengths in modeling speech signals. We introduce three strategies: (1) Transfer learning to initialize the speaker embedding network, improving generalization and reducing overfitting. (2) Knowledge distillation to train a more flexible speaker verification model, incorporating frame-level ASR loss as an auxiliary task. (3) A lightweight speaker adaptor for efficient feature conversion without altering the original ASR Conformer, allowing parallel ASR and speaker verification. Experiments on VoxCeleb show significant improvements: transfer learning yields a 0.48% EER, knowledge distillation results in a 0.43% EER, and the speaker adaptor approach, with just an added 4.92M parameters to a 130.94M-parameter model, achieves a 0.57% EER. Overall, our methods effectively transfer ASR capabilities to speaker verification tasks.
Code (0)
등록된 구현이 없습니다.
Tasks
Knowledge DistillationSpeaker VerificationTransfer LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Universal Pooling Method of Multi-layer Features from Pretrained Models for Speaker Verification
Recent advancements in automatic speaker verification (ASV) studies have been achieved by leveraging large-scale pretrained networks. In this study, we analyze the approaches toward such a paradigm and underline the sign…
Speaker VerificationText-Independent Speaker VerificationImproving Neural Diarization through Speaker Attribute Attractors and Local Dependency Modeling
In recent years, end-to-end approaches have made notable progress in addressing the challenge of speaker diarization, which involves segmenting and identifying speakers in multi-talker recordings. One such approach, Enco…
AttributeDecoderspeaker-diarizationSpeaker DiarizationImproving Speaker Verification with Self-Pretrained Transformer Models
Recently, fine-tuning large pre-trained Transformer models using downstream datasets has received a rising interest. Despite their success, it is still challenging to disentangle the benefits of large-scale datasets and …
Speaker VerificationOn Time Domain Conformer Models for Monaural Speech Separation in Noisy Reverberant Acoustic Environments
Speech separation remains an important topic for multi-speaker technology researchers. Convolution augmented transformers (conformers) have performed well for many speech processing tasks but have been under-researched f…
Computational EfficiencySpeech SeparationRethinking Leveraging Pre-Trained Multi-Layer Representations for Speaker Verification
Recent speaker verification studies have achieved notable success by leveraging layer-wise output from pre-trained Transformer models. However, few have explored the advancements in aggregating these multi-level features…
Speaker Verification