Reducing language context confusion for end-to-end code-switching automatic speech recognition
Code-switching deals with alternative languages in communication process. Training end-to-end (E2E) automatic speech recognition (ASR) systems for code-switching is especially challenging as code-switching training data are always insufficient to combat the increased multilingual context confusion due to the presence of more than one language. We propose a language-related attention mechanism to reduce multilingual context confusion for the E2E code-switching ASR model based on the Equivalence Constraint (EC) Theory. The linguistic theory requires that any monolingual fragment that occurs in the code-switching sentence must occur in one of the monolingual sentences. The theory establishes a bridge between monolingual data and code-switching data. We leverage this linguistics theory to design the code-switching E2E ASR model. The proposed model efficiently transfers language knowledge from rich monolingual data to improve the performance of the code-switching ASR model. We evaluate our model on ASRU 2019 Mandarin-English code-switching challenge dataset. Compared to the baseline model, our proposed model achieves a 17.12% relative error reduction.
Code (0)
등록된 구현이 없습니다.
Tasks
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Sentencespeech-recognitionSpeech RecognitionSimilar Papers 제목 키워드 기반
Reducing Language confusion for Code-switching Speech Recognition with Token-level Language Diarization
Code-switching (CS) refers to the phenomenon that languages switch within a speech signal and leads to language confusion for automatic speech recognition (ASR). This paper aims to address language confusion for improvin…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)speech-recognitionSpeech RecognitionLeveraging Language ID to Calculate Intermediate CTC Loss for Enhanced Code-Switching Speech Recognition
In recent years, end-to-end speech recognition has emerged as a technology that integrates the acoustic, pronunciation dictionary, and language model components of the traditional Automatic Speech Recognition model. It i…
Automatic Speech RecognitionLanguage IdentificationLanguage ModelingLanguage Modelling+2Language Confusion Gate: Language-Aware Decoding Through Model Self-Distillation
Large language models (LLMs) often experience language confusion, which is the unintended mixing of languages during text generation. Current solutions to this problem either necessitate model retraining or cannot differ…
Text GenerationAligning Speech to Languages to Enhance Code-switching Speech Recognition
Code-switching (CS) refers to the switching of languages within a speech signal and results in language confusion for automatic speech recognition (ASR). To address language confusion, we introduce a novel language align…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)DecoderLanguage Identification+4Language-Aware Token Boosting: LLM Language Confusion Reduction Without Tuning
Large language models (LLMs) sometimes exhibit language confusion when generating non-English text. Existing approaches typically rely on fine-tuning to mitigate this issue. In contrast, we propose a tuning-free paradigm…