Evaluating zero-shot transfers and multilingual models for dependency parsing and POS tagging within the low-resource language family Tupían
This work presents two experiments with the goal of replicating the transferability of dependency parsers and POS taggers trained on closely related languages within the low-resource language family Tupían. The experiments include both zero-shot settings as well as multilingual models. Previous studies have found that even a comparably small treebank from a closely related language will improve sequence labelling considerably in such cases. Results from both POS tagging and dependency parsing confirm previous evidence that the closer the phylogenetic relation between two languages, the better the predictions for sequence labelling tasks get. In many cases, the results are improved if multiple languages from the same family are combined. This suggests that in addition to leveraging similarity between two related languages, the incorporation of multiple languages of the same family might lead to better results in transfer learning for NLP applications.
Code (0)
등록된 구현이 없습니다.
Tasks
Dependency ParsingPOSPOS TaggingTransfer LearningSimilar Papers 제목 키워드 기반
From Zero to Hero: On the Limitations of Zero-Shot Language Transfer with Multilingual Transformers
Massively multilingual transformers (MMTs) pretrained via language modeling (e.g., mBERT, XLM-R) have become a default paradigm for zero-shot language transfer in NLP, offering unmatched transfer performance. Current eva…
Cross-Lingual Word EmbeddingsDependency ParsingLanguage ModelingLanguage Modelling+5Languages Transferred Within the Encoder: On Representation Transfer in Zero-Shot Multilingual Translation
Understanding representation transfer in multilingual neural machine translation can reveal the representational issue causing the zero-shot translation deficiency. In this work, we introduce the identity pair, a sentenc…
Contrastive LearningDecoderMachine TranslationSentence+1From English To Foreign Languages: Transferring Pre-trained Language Models
Pre-trained models have demonstrated their effectiveness in many downstream natural language processing (NLP) tasks. The availability of multilingual pre-trained models enables zero-shot transfer of NLP tasks from high r…
Dependency ParsingGPUNatural Language InferenceMultilingualism Encourages Recursion: a Transfer Study with mBERT
The present work constitutes an attempt to investigate the relational structures learnt by mBERT, a multilingual transformer-based network, with respect to different cross-linguistic regularities proposed in the fields o…
Zero-shot Dependency Parsing with Pre-trained Multilingual Sentence Representations
We investigate whether off-the-shelf deep bidirectional sentence representations trained on a massively multilingual corpus (multilingual BERT) enable the development of an unsupervised universal dependency parser. This …
Dependency ParsingSentenceTranslation