Reducing Language confusion for Code-switching Speech Recognition with Token-level Language Diarization

2022-10-26 · Hexin Liu, HaiHua Xu, Leibny Paola Garcia, Andy W. H. Khong, Yi He, Sanjeev Khudanpur

Code-switching (CS) refers to the phenomenon that languages switch within a speech signal and leads to language confusion for automatic speech recognition (ASR). This paper aims to address language confusion for improving CS-ASR from two perspectives: incorporating and disentangling language information. We incorporate language information in the CS-ASR model by dynamically biasing the model with token-level language posteriors which are outputs of a sequence-to-sequence auxiliary language diarization module. In contrast, the disentangling process reduces the difference between languages via adversarial training so as to normalize two languages. We conduct the experiments on the SEAME dataset. Compared to the baseline model, both the joint optimization with LD and the language posterior bias achieve performance improvement. The comparison of the proposed methods indicates that incorporating language information is more effective than disentangling for reducing language confusion in CS speech.

📄 PDF Abstract BibTeX arXiv:2210.14567

Code (1)

Lhx94As/reducing_language_confusion 공식 구현 pytorch

Tasks

Automatic Speech RecognitionAutomatic Speech Recognition (ASR)speech-recognition Speech Recognition

Reducing language context confusion for end-to-end code-switching automatic speech recognition

2022-01-28 · Shuai Zhang, Jiangyan Yi, Zhengkun Tian, JianHua Tao 외

Code-switching deals with alternative languages in communication process. Training end-to-end (E2E) automatic speech recognition (ASR) systems for code-switching is especially challenging as code-switching training data …

Automatic Speech Recognition Automatic Speech Recognition (ASR)Sentence speech-recognition+1

Leveraging Language ID to Calculate Intermediate CTC Loss for Enhanced Code-Switching Speech Recognition

2023-12-15 · Tzu-Ting Yang, Hsin-Wei Wang, Berlin Chen

In recent years, end-to-end speech recognition has emerged as a technology that integrates the acoustic, pronunciation dictionary, and language model components of the traditional Automatic Speech Recognition model. It i…

Automatic Speech Recognition Language Identification Language Modeling Language Modelling+2

Aligning Speech to Languages to Enhance Code-switching Speech Recognition

2024-03-09 · Hexin Liu, Xiangyu Zhang, Haoyang Zhang, Leibny Paola Garcia 외

Code-switching (CS) refers to the switching of languages within a speech signal and results in language confusion for automatic speech recognition (ASR). To address language confusion, we introduce a novel language align…

Automatic Speech Recognition Automatic Speech Recognition (ASR)Decoder Language Identification+4

Code-Switching without Switching: Language Agnostic End-to-End Speech Translation

2022-10-04 · Christian Huber, Enes Yavuz Ugan, Alexander Waibel

We propose a) a Language Agnostic end-to-end Speech Translation model (LAST), and b) a data augmentation strategy to increase code-switching (CS) performance. With increasing globalization, multiple languages are increas…

Data Augmentation speech-recognition Speech Recognition Translation

Optimizing Bilingual Neural Transducer with Synthetic Code-switching Text Generation

2022-10-21 · Thien Nguyen, Nathalie Tran, Liuhui Deng, Thiago Fraga da Silva 외

Code-switching describes the practice of using more than one language in the same sentence. In this study, we investigate how to optimize a neural transducer based bilingual automatic speech recognition (ASR) model for c…

Automatic Speech Recognition Automatic Speech Recognition (ASR)Sentence speech-recognition+2