Evaluating a Dependency Parser on DeReKo
We evaluate a graph-based dependency parser on DeReKo, a large corpus of contemporary German. The dependency parser is trained on the German dataset from the SPMRL 2014 Shared Task which contains text from the news domain, whereas DeReKo also covers other domains including fiction, science, and technology. To avoid the need for costly manual annotation of the corpus, we use the parser{'}s probability estimates for unlabeled and labeled attachment as main evaluation criterion. We show that these probability estimates are highly correlated with the actual attachment scores on a manually annotated test set. On this basis, we compare estimated parsing scores for the individual domains in DeReKo, and show that the scores decrease with increasing distance of a domain to the training corpus.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Named Entity Tagging a Very Large Unbalanced Corpus: Training and Evaluating NE Classifiers
We describe a systematic and application-oriented approach to training and evaluating named entity recognition and classification (NERC) systems, the purpose of which is to identify an optimal system and to train an opti…
ChunkingMachine Translationnamed-entity-recognitionNamed Entity Recognition+3RU-EVAL-2012: Evaluating Dependency Parsers for Russian
A Gold Standard Dependency Corpus for English
We present a gold standard annotation of syntactic dependencies in the English Web Treebank corpus using the Stanford Dependencies formalism. This resource addresses the lack of a gold standard dependency treebank for En…
Sentiment AnalysisParsing Any Domain English text to CoNLL dependencies
It is well known that accuracies of statistical parsers trained over Penn Treebank on test sets drawn from the same corpus tend to be overestimates of their actual parsing performance. This gives rise to the need for eva…
BenchmarkingDependency ParsingMachine TranslationQuestion Answering+2Improving a Strong Neural Parser with Conjunction-Specific Features
While dependency parsers reach very high overall accuracy, some dependency relations are much harder than others. In particular, dependency parsers perform poorly in coordination construction (i.e., correctly attaching t…
Dependency Parsing