Training a code-switching language model with monolingual data
A lack of code-switching data complicates the training of code-switching (CS) language models. We propose an approach to train such CS language models on monolingual data only. By constraining and normalizing the output projection matrix in RNN-based language models, we bring embeddings of different languages closer to each other. Numerical and visualization results show that the proposed approaches remarkably improve the performance of CS language models trained on monolingual data. The proposed approaches are comparable or even better than training CS language models with artificially generated CS data. We additionally use unsupervised bilingual word translation to analyze whether semantically equivalent words in different languages are mapped together.
Code (0)
등록된 구현이 없습니다.
Tasks
Language ModelingLanguage ModellingTranslationWord TranslationSimilar Papers 제목 키워드 기반
Reducing language context confusion for end-to-end code-switching automatic speech recognition
Code-switching deals with alternative languages in communication process. Training end-to-end (E2E) automatic speech recognition (ASR) systems for code-switching is especially challenging as code-switching training data …
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Sentencespeech-recognition+1Constrained Output Embeddings for End-to-End Code-Switching Speech Recognition with Only Monolingual Data
The lack of code-switch training data is one of the major concerns in the development of end-to-end code-switching automatic speech recognition (ASR) models. In this work, we propose a method to train an improved end-to-…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)speech-recognitionSpeech RecognitionCode Switching Language Model Using Monolingual Training Data
Training a code-switching (CS) language model using only monolingual data is still an ongoing research problem. In this paper, a CS language model is trained using only monolingual training data. As recurrent neural netw…
Language ModelingLanguage ModellingCode-Switching Curriculum Learning for Multilingual Transfer in LLMs
Large language models (LLMs) now exhibit near human-level performance in various tasks, but their performance drops drastically after a handful of high-resource languages due to the imbalance in pre-training data. Inspir…
Cross-Lingual TransferLanguage AcquisitionSafety AlignmentSentenceLearning to Recognize Code-switched Speech Without Forgetting Monolingual Speech Recognition
Recently, there has been significant progress made in Automatic Speech Recognition (ASR) of code-switched speech, leading to gains in accuracy on code-switched datasets in many language pairs. Code-switched speech co-occ…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)speech-recognitionSpeech Recognition