Multi-teacher Distillation for Multilingual Spelling Correction
Accurate spelling correction is a critical step in modern search interfaces, especially in an era of mobile devices and speech-to-text interfaces. For services that are deployed around the world, this poses a significant challenge for multilingual NLP: spelling errors need to be caught and corrected in all languages, and even in queries that use multiple languages. In this paper, we tackle this challenge using multi-teacher distillation. On our approach, a monolingual teacher model is trained for each language/locale, and these individual models are distilled into a single multilingual student model intended to serve all languages/locales. In experiments using open-source data as well as user data from a worldwide search service, we show that this leads to highly effective spelling correction models that can meet the tight latency requirements of deployed services.
Code (0)
등록된 구현이 없습니다.
Tasks
Multilingual NLPSpeech-to-TextSpelling CorrectionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
GitHub Typo Corpus: A Large-Scale Multilingual Dataset of Misspellings and Grammatical Errors
The lack of large-scale datasets has been a major hindrance to the development of NLP tasks such as spelling correction and grammatical error correction (GEC). As a complementary new resource for these tasks, we present …
Grammatical Error CorrectionSpelling CorrectionLanguage-Specialized Multi-Teacher On-Policy Distillation for Multilingual LLM-Based ASR
Modern LLM-based ASR systems have established multilingual capability as a standard feature, leveraging large-scale multilingual corpora and LLMs' cross-lingual knowledge to achieve competitive performance across multili…
Reinforcement LearningThe Privileged Students: On the Value of Initialization in Multilingual Knowledge Distillation
Knowledge distillation (KD) has proven to be a successful strategy to improve the performance of smaller models in many NLP tasks. However, most of the work in KD only explores monolingual scenarios. In this paper, we in…
Knowledge DistillationAMTSS: An Adaptive Multi-Teacher Single-Student Knowledge Distillation Framework For Multilingual Language Inference
Knowledge distillation is of key importance to launching multilingual pre-trained language models for real applications. To support cost-effective language inference in multilingual settings, we propose AMTSS, an adaptiv…
Knowledge DistillationMulti-Teacher Language-Aware Knowledge Distillation for Multilingual Speech Emotion Recognition
Speech Emotion Recognition (SER) is crucial for improving human-computer interaction. Despite strides in monolingual SER, extending them to build a multilingual system remains challenging. Our goal is to train a single m…
Emotion RecognitionKnowledge DistillationSpeech Emotion Recognition