Annotation Errors and NER: A Study with OntoNotes 5.0
Named Entity Recognition (NER) is a well-studied problem in NLP. However, there is much less focus on studying NER datasets, compared to developing new NER models. In this paper, we employed three simple techniques to detect annotation errors in the OntoNotes 5.0 corpus for English NER, which is the largest available NER corpus for English. Our techniques corrected ~10% of the sentences in train/dev/test data. In terms of entity mentions, we corrected the span and/or type of ~8% of mentions in the dataset, while adding/deleting/splitting/merging a few more. These are large numbers of changes, considering the size of OntoNotes. We used three NER libraries to train, evaluate and compare the models trained with the original and the re-annotated datasets, which showed an average improvement of 1.23% in overall F-scores, with large (>10%) improvements for some of the entity types. While our annotation error detection methods are not exhaustive and there is some manual annotation effort involved, they are largely language agnostic and can be employed with other NER datasets, and other sequence labelling tasks.
Code (0)
등록된 구현이 없습니다.
Tasks
named-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)NERMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Fine-grained Named Entity Annotation for Finnish
We introduce a corpus with fine-grained named entity annotation for Finnish, following the OntoNotes guidelines to create a resource that is cross-lingually compatible with existing annotations for other languages. We co…
NERLongtonotes: OntoNotes with Longer Coreference Chains
Ontonotes has served as the most important benchmark for coreference resolution. However, for ease of annotation, several long documents in Ontonotes were split into smaller parts. In this work, we build a corpus of core…
coreference-resolutionCoreference ResolutionFantasyCoref: Coreference Resolution on Fantasy Literature Through Omniscient Writer’s Point of View
This paper presents a new corpus and annotation guideline for a novel coreference resolution task on fictional texts, and analyzes its unique characteristics. FantasyCoref contains 211 stories of Grimms’ Fairy Tales and …
coreference-resolutionCoreference ResolutionAnalyzing Coreference and Bridging in Product Reviews
Product reviews may have complex discourse including coreference and bridging relations to a main product, competing products, and interacting products. Current approaches to aspect-based sentiment analysis (ABSA) and op…
Aspect-Based Sentiment AnalysisAspect-Based Sentiment Analysis (ABSA)Opinion SummarizationSentiment AnalysisCan we Fix the Scope for Coreference? Problems and Solutions for Benchmarks beyond OntoNotes
Current work on automatic coreference resolution has focused on the OntoNotes benchmark dataset, due to both its size and consistency. However many aspects of the OntoNotes annotation scheme are not well understood by NL…
coreference-resolutionCoreference Resolution