paper-with-me

홈 › Papers

CL-MASR: A Continual Learning Benchmark for Multilingual ASR

2023-10-25 · Luca Della Libera, Pooneh Mousavi, Salah Zaiem, Cem Subakan, Mirco Ravanelli

Modern multilingual automatic speech recognition (ASR) systems like Whisper have made it possible to transcribe audio in multiple languages with a single model. However, current state-of-the-art ASR models are typically evaluated on individual languages or in a multi-task setting, overlooking the challenge of continually learning new languages. There is insufficient research on how to add new languages without losing valuable information from previous data. Furthermore, existing continual learning benchmarks focus mostly on vision and language tasks, leaving continual learning for multilingual ASR largely unexplored. To bridge this gap, we propose CL-MASR, a benchmark designed for studying multilingual ASR in a continual learning setting. CL-MASR provides a diverse set of continual learning methods implemented on top of large-scale pretrained ASR models, along with common metrics to assess the effectiveness of learning new languages while addressing the issue of catastrophic forgetting. To the best of our knowledge, CL-MASR is the first continual learning benchmark for the multilingual ASR task. The code is available at https://github.com/speechbrain/benchmarks.

📄 PDF Abstract BibTeX arXiv:2310.16931

Code (1)

speechbrain/benchmarks 공식 구현 pytorch

Tasks

Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Continual Learningspeech-recognitionSpeech Recognition

Methods 이 논문이 사용한 방법론

SET Dynamic Sparse Training method where weight mask is updated randomly periodically
Focus 설명 없음

Similar Papers 제목 키워드 기반

Continual Learning Optimizations for Auto-regressive Decoder of Multilingual ASR systems

2024-07-04 · Chin Yuen Kwok, Jia Qi Yip, Eng Siong Chng

Continual Learning (CL) involves fine-tuning pre-trained models with new data while maintaining the performance on the pre-trained data. This is particularly relevant for expanding multilingual ASR (MASR) capabilities. H…

Continual LearningDecoder

A Parameter-efficient Language Extension Framework for Multilingual ASR

2024-06-10 · Wei Liu, Jingyong Hou, Dong Yang, Muyong Cao 외

Covering all languages with a multilingual speech recognition model (MASR) is very difficult. Performing language extension on top of an existing MASR is a desirable choice. In this study, the MASR continual learning pro…

Continual Learningparameter-efficient fine-tuningspeech-recognitionSpeech Recognition

Configurable Multilingual ASR with Speech Summary Representations

2024-10-06 · Harrison Zhu, Ivan Fung, Yingke Zhu, Lahiru Samarakoon

Approximately half of the world's population is multilingual, making multilingual ASR (MASR) essential. Deploying multiple monolingual models is challenging when the ground-truth language is unknown in advance. This moti…

BLR-MoE: Boosted Language-Routing Mixture of Experts for Domain-Robust Multilingual E2E ASR

2025-01-22 · Guodong Ma, Wenxuan Wang, Lifeng Zhou, Yuting Yang 외

Recently, the Mixture of Expert (MoE) architecture, such as LR-MoE, is often used to alleviate the impact of language confusion on the multilingual ASR (MASR) task. However, it still faces language confusion issues, espe…

Mixture-of-Experts

A Language-Agnostic Hierarchical LoRA-MoE Architecture for CTC-based Multilingual ASR

2026-01-02 · Yuang Zheng, Dongxu Chen, Yuxiang Mei, Dongxing Xu 외 arxiv

Large-scale multilingual ASR (mASR) models such as Whisper achieve strong performance but incur high computational and latency costs, limiting their deployment on resource-constrained edge devices. In this study, we prop…

Domain Adaptation