Spectral-Aware Low-Rank Adaptation for Speaker Verification
Previous research has shown that the principal singular vectors of a pre-trained model's weight matrices capture critical knowledge. In contrast, those associated with small singular values may contain noise or less reliable information. As a result, the LoRA-based parameter-efficient fine-tuning (PEFT) approach, which does not constrain the use of the spectral space, may not be effective for tasks that demand high representation capacity. In this study, we enhance existing PEFT techniques by incorporating the spectral information of pre-trained weight matrices into the fine-tuning process. We investigate spectral adaptation strategies with a particular focus on the additive adjustment of top singular vectors. This is accomplished by applying singular value decomposition (SVD) to the pre-trained weight matrices and restricting the fine-tuning within the top spectral space. Extensive speaker verification experiments on VoxCeleb1 and CN-Celeb1 demonstrate enhanced tuning performance with the proposed approach. Code is released at https://github.com/lizhepolyu/SpectralFT.
Code (1)
Tasks
parameter-efficient fine-tuningSpeaker VerificationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Channel Adaptation for Speaker Verification Using Optimal Transport with Pseudo Label
Domain gap often degrades the performance of speaker verification (SV) systems when the statistical distributions of training data and real-world test speech are mismatched. Channel variation, a primary factor causing th…
Domain AdaptationPseudo LabelSpeaker VerificationUnsupervised Domain AdaptationLarge Audio Language Models for Spoofing-Aware Speaker Verification
Recent advances in text-to-speech and voice cloning make high-quality spoofing inexpensive and scalable, threatening voice authentication systems, especially automatic speaker verification (ASV). Existing defenses mainly…
Speaker VerificationDeepFake DetectionRobust Support Vector Machines for Speaker Verification Task
An important step in speaker verification is extracting features that best characterize the speaker voice. This paper investigates a front-end processing that aims at improving the performance of speaker verification bas…
Dimensionality ReductionSpeaker VerificationMulti-objective Progressive Clustering for Semi-supervised Domain Adaptation in Speaker Verification
Utilizing the pseudo-labeling algorithm with large-scale unlabeled data becomes crucial for semi-supervised domain adaptation in speaker verification tasks. In this paper, we propose a novel pseudo-labeling method named …
ClusteringDenoisingDomain AdaptationSemi-supervised Domain Adaptation+1What and When to Learn: CURriculum Ranking Loss for Large-Scale Speaker Verification
Speaker verification at large scale remains an open challenge as fixed-margin losses treat all samples equally regardless of quality. We hypothesize that mislabeled or degraded samples introduce noisy gradients that disr…
Speaker Verification