Zero-shot Dependency Parsing with Pre-trained Multilingual Sentence Representations
We investigate whether off-the-shelf deep bidirectional sentence representations trained on a massively multilingual corpus (multilingual BERT) enable the development of an unsupervised universal dependency parser. This approach only leverages a mix of monolingual corpora in many languages and does not require any translation data making it applicable to low-resource languages. In our experiments we outperform the best CoNLL 2018 language-specific systems in all of the shared task's six truly low-resource languages while using a single system. However, we also find that (i) parsing accuracy still varies dramatically when changing the training languages and (ii) in some target languages zero-shot transfer fails under all tested conditions, raising concerns on the 'universality' of the whole approach.
Code (0)
등록된 구현이 없습니다.
Tasks
Dependency ParsingSentenceTranslationSimilar Papers 제목 키워드 기반
UDapter: Typology-based Language Adapters for Multilingual Dependency Parsing and Sequence Labeling
Recent advances in multilingual language modeling have brought the idea of a truly universal parser closer to reality. However, such models are still not immune to the “curse of multilinguality”: Cross-language interfere…
Dependency ParsingLanguage ModelingLanguage ModellingMorphological Tagging+3Zero-Shot Dependency Parsing with Worst-Case Aware Automated Curriculum Learning
Large multilingual pretrained language models such as mBERT and XLM-RoBERTa have been found to be surprisingly effective for cross-lingual transfer of syntactic parsing models (Wu and Dredze 2019), but only between relat…
Cross-Lingual TransferDependency ParsingMulti-Task LearningZero-Shot Dependency Parsing with Worst-Case Aware Automated Curriculum Learning
Large multilingual pretrained language models such as mBERT and XLM-RoBERTa have been found to be surprisingly effective for cross-lingual transfer of syntactic parsing models Wu and Dredze (2019), but only between relat…
Cross-Lingual TransferDependency ParsingMulti-Task LearningEvaluating zero-shot transfers and multilingual models for dependency parsing and POS tagging within the low-resource language family Tupían
This work presents two experiments with the goal of replicating the transferability of dependency parsers and POS taggers trained on closely related languages within the low-resource language family Tupían. The experimen…
Dependency ParsingPOSPOS TaggingTransfer LearningChatGPT is a Potential Zero-Shot Dependency Parser
Pre-trained language models have been widely used in dependency parsing task and have achieved significant improvements in parser performance. However, it remains an understudied question whether pre-trained language mod…
Dependency Parsing