Negation-Instance Based Evaluation of End-to-End Negation Resolution
In this paper, we revisit the task of negation resolution, which includes the subtasks of cue detection (e.g. "not", "never") and scope resolution. In the context of previous shared tasks, a variety of evaluation metrics have been proposed. Subsequent works usually use different subsets of these, including variations and custom implementations, rendering meaningful comparisons between systems difficult. Examining the problem both from a linguistic perspective and from a downstream viewpoint, we here argue for a negation-instance based approach to evaluating negation resolution. Our proposed metrics correspond to expectations over per-instance scores and hence are intuitively interpretable. To render research comparable and to foster future work, we provide results for a set of current state-of-the-art systems for negation resolution on three English corpora, and make our implementation of the evaluation scripts publicly available.
Code (1)
Tasks
NegationSimilar Papers 제목 키워드 기반
Learning Interpretable Negation Rules via Weak Supervision at Document Level: A Reinforcement Learning Approach
Negation scope detection is widely performed as a supervised learning task which relies upon negation labels at word level. This suffers from two key drawbacks: (1) such granular annotations are costly and (2) highly sub…
NegationReinforcement LearningSentiment AnalysisAn Analysis of Negation in Natural Language Understanding Corpora
This paper analyzes negation in eight popular corpora spanning six natural language understanding tasks. We show that these corpora have few negations compared to general-purpose English, and that the few negations in th…
Natural Language UnderstandingNegationNegation Triplet Extraction with Syntactic Dependency and Semantic Consistency
Previous works of negation understanding mainly focus on negation cue detection and scope resolution, without identifying negation subject which is also significant to the downstream tasks. In this paper, we propose a ne…
DecoderLanguage ModellingMulti-Task LearningNegation+2Resolving Legalese: A Multilingual Exploration of Negation Scope Resolution in Legal Documents
Resolving the scope of a negation within a sentence is a challenging NLP task. The complexity of legal texts and the lack of annotated in-domain negation corpora pose challenges for state-of-the-art (SotA) models when pe…
NegationNegation Scope ResolutionSentenceMultitask Learning of Negation and Speculation using Transformers
Detecting negation and speculation in language has been a task of considerable interest to the biomedical community, as it is a key component of Information Extraction systems from Biomedical documents. Prior work has in…
NegationNegation and Speculation Scope resolutionNegation DetectionNegation Scope Resolution+2