Out-of-Domain Evaluation of Finnish Dependency Parsing
The prevailing practice in the academia is to evaluate the model performance on in-domain evaluation data typically set aside from the training corpus. However, in many real world applications the data on which the model is applied may very substantially differ from the characteristics of the training data. In this paper, we focus on Finnish out-of-domain parsing by introducing a novel UD Finnish-OOD out-of-domain treebank including five very distinct data sources (web documents, clinical, online discussions, tweets, and poetry), and a total of 19,382 syntactic words in 2,122 sentences released under the Universal Dependencies framework. Together with the new treebank, we present extensive out-of-domain parsing evaluation utilizing the available section-level information from three different Finnish UD treebanks (TDT, PUD, OOD). Compared to the previously existing treebanks, the new Finnish-OOD is shown include sections more challenging for the general parser, creating an interesting evaluation setting and yielding valuable information for those applying the parser outside of its training domain.
Code (1)
Tasks
Dependency ParsingSimilar Papers 제목 키워드 기반
Joint Morphological and Syntactic Analysis for Richly Inflected Languages
Joint morphological and syntactic analysis has been proposed as a way of improving parsing accuracy for richly inflected languages. Starting from a transition-based model for joint part-of-speech tagging and dependency p…
Dependency ParsingPart-Of-Speech TaggingMind the Gap: Data Enrichment in Dependency Parsing of Elliptical Constructions
In this paper, we focus on parsing rare and non-trivial constructions, in particular ellipsis. We report on several experiments in enrichment of training data for this specific construction, evaluated on five languages: …
Dependency ParsingSemi-Supervised Methods for Out-of-Domain Dependency Parsing
Dependency parsing is one of the important natural language processing tasks that assigns syntactic trees to texts. Due to the wider availability of dependency corpora and improved parsing and machine learning techniques…
Dependency ParsingDependency Parsing Evaluation for Low-resource Spontaneous Speech
How well can a state-of-the-art parsing system, developed for the written domain, perform when applied to spontaneous speech data involving different interlocutors? This study addresses this question in a low-resource se…
Dependency ParsingAn Empirical Exploration of Local Ordering Pre-training for Structured Prediction
Recently, pre-training contextualized encoders with language model (LM) objectives has been shown an effective semi-supervised method for structured prediction. In this work, we empirically explore an alternative pre-tra…
Dependency ParsingLanguage ModelingLanguage ModellingNER+3