On the Benefit of Syntactic Supervision for Cross-lingual Transfer in Semantic Role Labeling
Although recent developments in neural architectures and pre-trained representations have greatly increased state-of-the-art model performance on fully-supervised semantic role labeling (SRL), the task remains challenging for languages where supervised SRL training data are not abundant. Cross-lingual learning can improve performance in this setting by transferring knowledge from high-resource languages to low-resource ones. Moreover, we hypothesize that annotations of syntactic dependencies can be leveraged to further facilitate cross-lingual transfer. In this work, we perform an empirical exploration of the helpfulness of syntactic supervision for crosslingual SRL within a simple multitask learning scheme. With comprehensive evaluations across ten languages (in addition to English) and three SRL benchmark datasets, including both dependency- and span-based SRL, we show the effectiveness of syntactic supervision in low-resource scenarios.
Code (1)
Tasks
Cross-Lingual TransferSemantic Role LabelingSimilar Papers 제목 키워드 기반
Cross-Linguistic Syntactic Difference in Multilingual BERT: How Good is It and How Does It Affect Transfer?
Multilingual BERT (mBERT) has demonstrated considerable cross-lingual syntactic ability, whereby it enables effective zero-shot cross-lingual transfer of syntactic knowledge. The transfer is more successful between some …
Cross-Lingual TransferDiversityZero-Shot Cross-Lingual TransferIncorporating Lexical and Syntactic Knowledge for Unsupervised Cross-Lingual Transfer
Unsupervised cross-lingual transfer involves transferring knowledge between languages without explicit supervision. Although numerous studies have been conducted to improve performance in such tasks by focusing on cross-…
Cross-Lingual TransferGraph Attentionnamed-entity-recognitionNamed Entity Recognition+6Finding Universal Grammatical Relations in Multilingual BERT
Recent work has found evidence that Multilingual BERT (mBERT), a transformer-based multilingual masked language model, is capable of zero-shot cross-lingual transfer, suggesting that some aspects of its representations a…
Cross-Lingual TransferLanguage ModelingLanguage ModellingZero-Shot Cross-Lingual TransferIsomorphic Transfer of Syntactic Structures in Cross-Lingual NLP
The transfer or share of knowledge between languages is a potential solution to resource scarcity in NLP. However, the effectiveness of cross-lingual transfer can be challenged by variation in syntactic structures. Frame…
Cross-Lingual TransferMachine TranslationRepresentation LearningSentence+3Same Neurons, Different Languages: Probing Morphosyntax in Multilingual Pre-trained Models
The success of multilingual pre-trained models in transferring knowledge cross-lingually is underpinned by their ability to learn representations shared by multiple languages even in absence of any explicit supervision. …