Joint unsupervised and supervised learning for context-aware language identification
Language identification (LID) recognizes the language of a spoken utterance automatically. According to recent studies, LID models trained with an automatic speech recognition (ASR) task perform better than those trained with a LID task only. However, we need additional text labels to train the model to recognize speech, and acquiring the text labels is a cost high. In order to overcome this problem, we propose context-aware language identification using a combination of unsupervised and supervised learning without any text labels. The proposed method learns the context of speech through masked language modeling (MLM) loss and simultaneously trains to determine the language of the utterance with supervised learning loss. The proposed joint learning was found to reduce the error rate by 15.6% compared to the same structure model trained by supervised-only learning on a subset of the VoxLingua107 dataset consisting of sub-three-second utterances in 11 languages.
Code (0)
등록된 구현이 없습니다.
Tasks
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Language IdentificationLanguage ModelingLanguage ModellingMasked Language Modelingspeech-recognitionSpeech RecognitionSimilar Papers 제목 키워드 기반
Large (Vision) Language Models are Unsupervised In-Context Learners
Recent advances in large language and vision-language models have enabled zero-shot inference, allowing models to solve new tasks without task-specific training. Various adaptation techniques such as prompt engineering, …
GSM8KIn-Context LearningMathPrompt EngineeringUnsupervised Extractive Summarization of Emotion Triggers
Understanding what leads to emotions during large-scale crises is important as it can provide groundings for expressed emotions and subsequently improve the understanding of ongoing disasters. Recent approaches trained s…
Abstractive Text SummarizationExtractive SummarizationUnsupervised Extractive SummarizationContext-Aware Semantic Similarity Measurement for Unsupervised Word Sense Disambiguation
The issue of word sense ambiguity poses a significant challenge in natural language processing due to the scarcity of annotated data to feed machine learning models to face the challenge. Therefore, unsupervised word sen…
Semantic SimilaritySemantic Textual SimilarityWord Sense DisambiguationAn Unsupervised Character-Aware Neural Approach to Word and Context Representation Learning
In the last few years, neural networks have been intensively used to develop meaningful distributed representations of words and contexts around them. When these representations, also known as "embeddings", are learned f…
Representation LearningUnsupervised Pose-Aware Part Decomposition for 3D Articulated Objects
Articulated objects exist widely in the real world. However, previous 3D generative methods for unsupervised part decomposition are unsuitable for such objects, because they assume a spatially fixed part location, result…
Pose Estimation