Improving Transformer-based Networks With Locality For Automatic Speaker Verification
Recently, Transformer-based architectures have been explored for speaker embedding extraction. Although the Transformer employs the self-attention mechanism to efficiently model the global interaction between token embeddings, it is inadequate for capturing short-range local context, which is essential for the accurate extraction of speaker information. In this study, we enhance the Transformer with the enhanced locality modeling in two directions. First, we propose the Locality-Enhanced Conformer (LE-Confomer) by introducing depth-wise convolution and channel-wise attention into the Conformer blocks. Second, we present the Speaker Swin Transformer (SST) by adapting the Swin Transformer, originally proposed for vision tasks, into speaker embedding network. We evaluate the proposed approaches on the VoxCeleb datasets and a large-scale Microsoft internal multilingual (MS-internal) dataset. The proposed models achieve 0.75% EER on VoxCeleb 1 test set, outperforming the previously proposed Transformer-based models and CNN-based models, such as ResNet34 and ECAPA-TDNN. When trained on the MS-internal dataset, the proposed models achieve promising results with 14.6% relative reduction in EER over the Res2Net50 model.
Code (0)
등록된 구현이 없습니다.
Tasks
Speaker VerificationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
A New Perspective on Speaker Verification: Joint Modeling with DFSMN and Transformer
Speaker verification is to judge the similarity between two unknown voices in an open set, where the ideal speaker embedding should be able to condense discriminant information into a compact utterance-level representati…
Speaker VerificationContrastive Predictive Coding Based Feature for Automatic Speaker Verification
This thesis describes our ongoing work on Contrastive Predictive Coding (CPC) features for speaker verification. CPC is a recently proposed representation learning framework based on predictive coding and noise contrasti…
Representation LearningSpeaker VerificationEvolutionary Algorithm Enhanced Neural Architecture Search for Text-Independent Speaker Verification
State-of-the-art speaker verification models are based on deep learning techniques, which heavily depend on the handdesigned neural architectures from experts or engineers. We borrow the idea of neural architecture searc…
Neural Architecture SearchSpeaker VerificationText-Independent Speaker VerificationS-vectors and TESA: Speaker Embeddings and a Speaker Authenticator Based on Transformer Encoder
One of the most popular speaker embeddings is x-vectors, which are obtained from an architecture that gradually builds a larger temporal context with layers. In this paper, we propose to derive speaker embeddings from Tr…
SentenceSpeaker VerificationAnalysis of Speech Temporal Dynamics in the Context of Speaker Verification and Voice Anonymization
In this paper, we investigate the impact of speech temporal dynamics in application to automatic speaker verification and speaker voice anonymization tasks. We propose several metrics to perform automatic speaker verific…
Speaker Verification