Exploring Cross-Lingual Transfer of Morphological Knowledge In Sequence-to-Sequence Models
Multi-task training is an effective method to mitigate the data sparsity problem. It has recently been applied for cross-lingual transfer learning for paradigm completion{---}the task of producing inflected forms of lemmata{---}with sequence-to-sequence networks. However, it is still vague how the model transfers knowledge across languages, as well as if and which information is shared. To investigate this, we propose a set of data-dependent experiments using an existing encoder-decoder recurrent neural network for the task. Our results show that indeed the performance gains surpass a pure regularization effect and that knowledge about language and morphology can be transferred.
Code (0)
등록된 구현이 없습니다.
Tasks
Cross-Lingual TransferDecoderLanguage ModelingLanguage ModellingMulti-Task LearningTransfer LearningSimilar Papers 제목 키워드 기반
Cross-lingual, Character-Level Neural Morphological Tagging
Even for common NLP tasks, sufficient supervision is not available in many languages -- morphological tagging is no exception. In the work presented here, we explore a transfer learning scheme, whereby we train character…
Morphological TaggingTransfer LearningExploring Cross-Lingual Knowledge Transfer via Transliteration-Based MLM Fine-Tuning for Critically Low-resource Chakma Language
As an Indo-Aryan language with limited available data, Chakma remains largely underrepresented in language models. In this work, we introduce a novel corpus of contextually coherent Bangla-transliterated Chakma, curated …
Transfer LearningEvaluating morphological typology in zero-shot cross-lingual transfer
Cross-lingual transfer has improved greatly through multi-lingual language model pretraining, reducing the need for parallel data and increasing absolute performance. However, this progress has also brought to light the …
Cross-Lingual TransferLanguage ModelingLanguage ModellingPart-Of-Speech Tagging+4The SIGMORPHON 2019 Shared Task: Morphological Analysis in Context and Cross-Lingual Transfer for Inflection
The SIGMORPHON 2019 shared task on cross-lingual transfer and contextual analysis in morphology examined transfer learning of inflection between 100 language pairs, as well as contextual lemmatization and morphosyntactic…
Cross-Lingual TransferLemmatizationMorphological AnalysisMorphological Inflection+1Cross-lingual Character-Level Neural Morphological Tagging
Even for common NLP tasks, sufficient supervision is not available in many languages {--} morphological tagging is no exception. In the work presented here, we explore a transfer learning scheme, whereby we train charact…
Language ModelingLanguage ModellingMorphological TaggingPart-Of-Speech Tagging+1