Code-Switching without Switching: Language Agnostic End-to-End Speech Translation
We propose a) a Language Agnostic end-to-end Speech Translation model (LAST), and b) a data augmentation strategy to increase code-switching (CS) performance. With increasing globalization, multiple languages are increasingly used interchangeably during fluent speech. Such CS complicates traditional speech recognition and translation, as we must recognize which language was spoken first and then apply a language-dependent recognizer and subsequent translation component to generate the desired target language output. Such a pipeline introduces latency and errors. In this paper, we eliminate the need for that, by treating speech recognition and translation as one unified end-to-end speech translation problem. By training LAST with both input languages, we decode speech into one target language, regardless of the input language. LAST delivers comparable recognition and speech translation accuracy in monolingual usage, while reducing latency and error rate considerably when CS is observed.
Code (0)
등록된 구현이 없습니다.
Tasks
Data Augmentationspeech-recognitionSpeech RecognitionTranslationSimilar Papers 제목 키워드 기반
Code-switching Sentence Generation by Generative Adversarial Networks and its Application to Data Augmentation
Code-switching is about dealing with alternative languages in speech or text. It is partially speaker-depend and domain-related, so completely explaining the phenomenon by linguistic rules is challenging. Compared to mos…
Data AugmentationGenerative Adversarial NetworkSentenceHierarchical Meta-Embeddings for Code-Switching Named Entity Recognition
In countries that speak multiple main languages, mixing up different languages within a conversation is commonly called code-switching. Previous works addressing this challenge mainly focused on word-level aspects such a…
named-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)Word EmbeddingsOptimizing Bilingual Neural Transducer with Synthetic Code-switching Text Generation
Code-switching describes the practice of using more than one language in the same sentence. In this study, we investigate how to optimize a neural transducer based bilingual automatic speech recognition (ASR) model for c…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Sentencespeech-recognition+2Multilingual Transfer Learning for Code-Switched Language and Speech Neural Modeling
In this thesis, we address the data scarcity and limitations of linguistic theory by proposing language-agnostic multi-task training methods. First, we introduce a meta-learning-based approach, meta-transfer learning, in…
Data AugmentationLanguage ModellingMeta-LearningMulti-Task Learning+1Reducing language context confusion for end-to-end code-switching automatic speech recognition
Code-switching deals with alternative languages in communication process. Training end-to-end (E2E) automatic speech recognition (ASR) systems for code-switching is especially challenging as code-switching training data …
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Sentencespeech-recognition+1