A Probabilistic Annotation Model for Crowdsourcing Coreference
The availability of large scale annotated corpora for coreference is essential to the development of the field. However, creating resources at the required scale via expert annotation would be too expensive. Crowdsourcing has been proposed as an alternative; but this approach has not been widely used for coreference. This paper addresses one crucial hurdle on the way to make this possible, by introducing a new model of annotation for aggregating crowdsourced anaphoric annotations. The model is evaluated along three dimensions: the accuracy of the inferred mention pairs, the quality of the post-hoc constructed silver chains, and the viability of using the silver chains as an alternative to the expert-annotated chains in training a state of the art coreference system. The results suggest that our model can extract from crowdsourced annotations coreference chains of comparable quality to those obtained with expert annotation.
Code (0)
등록된 구현이 없습니다.
Tasks
Coreference ResolutionmodelQuestion AnsweringSimilar Papers 제목 키워드 기반
CoRefi: A Crowd Sourcing Suite for Coreference Annotation
Coreference annotation is an important, yet expensive and time consuming, task, which often involved expert annotators trained on complex decision guidelines. To enable cheaper and more efficient annotation, we present C…
CoRefi: A Crowd Sourcing Suite for Coreference Annotation
Coreference annotation is an important, yet expensive and time consuming, task, which often involved expert annotators trained on complex decision guidelines. To enable cheaper and more efficient annotation, we present C…
Coreference ResolutionCross Document Coreference ResolutionA Neural Model for Aggregating Coreference Annotation in Crowdsourcing
Coreference resolution is the task of identifying all mentions in a text that refer to the same real-world entity. Collecting sufficient labelled data from expert annotators to train a high-performance coreference resolu…
coreference-resolutionCoreference ResolutionA Mention-Pair Model of Annotation with Nonparametric User Communities
The availability of large datasets is essential for progress in coreference and other areas of NLP. Crowdsourcing has proven a viable alternative to expert annotation, offering similar quality for better scalability. How…
ezCoref: A Scalable Approach for Collecting Crowdsourced Annotations for Coreference Resolution
Large-scale high-quality corpora are critical for advancing research in coreference resolution. Coreference annotation is typically time consuming and expensive, since researchers generally hire expert annotators and tra…
coreference-resolutionCoreference Resolution