CSSL: Contrastive Self-Supervised Learning for Dependency Parsing on Relatively Free Word Ordered and Morphologically Rich Low Resource Languages
Neural dependency parsing has achieved remarkable performance for low resource morphologically rich languages. It has also been well-studied that morphologically rich languages exhibit relatively free word order. This prompts a fundamental investigation: Is there a way to enhance dependency parsing performance, making the model robust to word order variations utilizing the relatively free word order nature of morphologically rich languages? In this work, we examine the robustness of graph-based parsing architectures on 7 relatively free word order languages. We focus on scrutinizing essential modifications such as data augmentation and the removal of position encoding required to adapt these architectures accordingly. To this end, we propose a contrastive self-supervised learning method to make the model robust to word order variations. Furthermore, our proposed modification demonstrates a substantial average gain of 3.03/2.95 points in 7 relatively free word order languages, as measured by the UAS/LAS Score metric when compared to the best performing baseline.
Code (0)
등록된 구현이 없습니다.
Tasks
Data AugmentationDependency ParsingSelf-Supervised LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Exemplar-Based Contrastive Self-Supervised Learning with Few-Shot Class Incremental Learning
Humans are capable of learning new concepts from only a few (labeled) exemplars, incrementally and continually. This happens within the context that we can differentiate among the exemplars, and between the exemplars and…
class-incremental learningClass Incremental LearningData AugmentationFew-Shot Class-Incremental Learning+3Concept Representation Learning with Contrastive Self-Supervised Learning
Concept-oriented deep learning (CODL) is a general approach to meet the future challenges for deep learning: (1) learning with little or no external supervision, (2) coping with test examples that come from a different d…
class-incremental learningClass Incremental LearningContinual LearningDeep Learning+4Contrastive Self-supervised Learning for Graph Classification
Graph classification is a widely studied problem and has broad applications. In many real-world problems, the number of labeled graphs available for training classification models is limited, which renders these models p…
ClassificationData AugmentationGeneral ClassificationGraph Classification+1C3-DINO: Joint Contrastive and Non-contrastive Self-Supervised Learning for Speaker Verification
Self-supervised learning (SSL) has drawn an increased attention in the field of speech processing. Recent studies have demonstrated that contrastive learning is able to learn discriminative speaker embeddings in a self-s…
Contrastive LearningSelf-Supervised LearningSpeaker VerificationContrastive Continuity on Augmentation Stability Rehearsal for Continual Self-Supervised Learning
Self-supervised learning has attracted a lot of attention recently, which is able to learn powerful representations without any manual annotations. However, self-supervised learning needs to develop the ability to co…
Continual Self-Supervised LearningSelf-Supervised Learning