Leveraging Cross-Lingual Transfer Learning in Spoken Named Entity Recognition Systems
Recent Named Entity Recognition (NER) advancements have significantly enhanced text classification capabilities. This paper focuses on spoken NER, aimed explicitly at spoken document retrieval, an area not widely studied due to the lack of comprehensive datasets for spoken contexts. Additionally, the potential for cross-lingual transfer learning in low-resource situations deserves further investigation. In our study, we applied transfer learning techniques across Dutch, English, and German using both pipeline and End-to-End (E2E) approaches. We employed Wav2Vec2 XLS-R models on custom pseudo-annotated datasets to evaluate the adaptability of cross-lingual systems. Our exploration of different architectural configurations assessed the robustness of these systems in spoken NER. Results showed that the E2E model was superior to the pipeline model, particularly with limited annotation resources. Furthermore, transfer learning from German to Dutch improved performance by 7% over the standalone Dutch E2E system and 4% over the Dutch pipeline model. Our findings highlight the effectiveness of cross-lingual transfer in spoken NER and emphasize the need for additional data collection to improve these systems.
Code (1)
Tasks
Cross-Lingual Transfernamed-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)NERSpeech RecognitionSpoken Language Understandingtext-classificationText ClassificationTransfer LearningSimilar Papers 제목 키워드 기반
GL-CLeF: A Global–Local Contrastive Learning Framework for Cross-lingual Spoken Language Understanding
Due to high data demands of current methods, attention to zero-shot cross-lingual spoken language understanding (SLU) has grown, as such approaches greatly reduce human annotation effort. However, existing models solely …
Contrastive LearningCross-Lingual TransferSentenceSpoken Language UnderstandingGL-CLeF: A Global-Local Contrastive Learning Framework for Cross-lingual Spoken Language Understanding
Due to high data demands of current methods, attention to zero-shot cross-lingual spoken language understanding (SLU) has grown, as such approaches greatly reduce human annotation effort. However, existing models solely …
Contrastive LearningCross-Lingual TransferSentenceSpoken Language UnderstandingWhat Matters for Neural Cross-Lingual Named Entity Recognition: An Empirical Analysis
Building named entity recognition (NER) models for languages that do not have much training data is a challenging task. While recent work has shown promising results on cross-lingual transfer from high-resource languages…
Cross-Lingual NERCross-Lingual Transfernamed-entity-recognitionNamed Entity Recognition+3MasakhaNER 2.0: Africa-centric Transfer Learning for Named Entity Recognition
African languages are spoken by over a billion people, but are underrepresented in NLP research and development. The challenges impeding progress include the limited availability of annotated datasets, as well as a lack …
Cross-Lingual Transfernamed-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)+2Cross-lingual Transfer Learning with Data Selection for Large-Scale Spoken Language Understanding
A typical cross-lingual transfer learning approach boosting model performance on a language is to pre-train the model on all available supervised data from another language. However, in large-scale systems this leads to …
Cross-Lingual TransferLanguage ModelingLanguage ModellingSpoken Language Understanding+1