Exploring Efficient-tuning Methods in Self-supervised Speech Models
In this study, we aim to explore efficient tuning methods for speech self-supervised learning. Recent studies show that self-supervised learning (SSL) can learn powerful representations for different speech tasks. However, fine-tuning pre-trained models for each downstream task is parameter-inefficient since SSL models are notoriously large with millions of parameters. Adapters are lightweight modules commonly used in NLP to solve this problem. In downstream tasks, the parameters of SSL models are frozen, and only the adapters are trained. Given the lack of studies generally exploring the effectiveness of adapters for self-supervised speech tasks, we intend to fill this gap by adding various adapter modules in pre-trained speech SSL models. We show that the performance parity can be achieved with over 90% parameter reduction, and discussed the pros and cons of efficient tuning techniques. This is the first comprehensive investigation of various adapter types across speech tasks.
Code (0)
등록된 구현이 없습니다.
Tasks
Self-Supervised LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Less Forgetting for Better Generalization: Exploring Continual-learning Fine-tuning Methods for Speech Self-supervised Representations
Despite being trained on massive and diverse datasets, speech self-supervised encoders are generally used for downstream purposes as mere frozen feature extractors or model initializers before fine-tuning. The former sev…
Continual LearningDomain Generalizationspeech-recognitionSpeech RecognitionExploring WavLM on Speech Enhancement
There is a surge in interest in self-supervised learning approaches for end-to-end speech encoding in recent years as they have achieved great success. Especially, WavLM showed state-of-the-art performance on various spe…
Self-Supervised LearningSpeech Enhancementspeech-recognitionSpeech RecognitionImproving Multilingual Speech Models on ML-SUPERB 2.0: Fine-tuning with Data Augmentation and LID-Aware CTC
Multilingual speech processing with self-supervised or supervised pre-trained Speech Foundation Models (SFM) has achieved strong performance on tasks like Language Identification (LID) and Automatic Speech Recognition (A…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Data AugmentationLanguage Identification+2Exploring Self-Supervised Multi-view Contrastive Learning for Speech Emotion Recognition with Limited Annotations
Recent advancements in Deep and Self-Supervised Learning (SSL) have led to substantial improvements in Speech Emotion Recognition (SER) performance, reaching unprecedented levels. However, obtaining sufficient amounts of…
Contrastive LearningEmotion RecognitionSelf-Supervised LearningSpeech Emotion RecognitionRepresentative Subset Selection for Efficient Fine-Tuning in Self-Supervised Speech Recognition
Self-supervised speech recognition models require considerable labeled training data for learning high-fidelity representations for Automatic Speech Recognition (ASR) which is computationally demanding and time-consuming…
Active LearningAutomatic Speech RecognitionAutomatic Speech Recognition (ASR)speech-recognition+1