Crowd-Sourced Iterative Annotation for Narrative Summarization Corpora
We present an iterative annotation process for producing aligned, parallel corpora of abstractive and extractive summaries for narrative. Our approach uses a combination of trained annotators and crowd-sourcing, allowing us to elicit human-generated summaries and alignments quickly and at low cost. We use crowd-sourcing to annotate aligned phrases with the text-to-text generation techniques needed to transform each phrase into the other. We apply this process to a corpus of 476 personal narratives, which we make available on the Web.
Code (0)
등록된 구현이 없습니다.
Tasks
Abstractive Text SummarizationSentence CompressionText GenerationText SummarizationSimilar Papers 제목 키워드 기반
SNaC: Coherence Error Detection for Narrative Summarization
Progress in summarizing long texts is inhibited by the lack of appropriate evaluation frameworks. When a long summary must be produced to appropriately cover the facets of that text, that summary needs to present a coher…
BenchmarkingCoherence EvaluationDocument SummarizationPhrase Detectives Corpus 1.0 Crowdsourced Anaphoric Coreference.
Natural Language Engineering tasks require large and complex annotated datasets to build more advanced models of language. Corpora are typically annotated by several experts to create a gold standard; however, there are …
text annotationIDN-Sum: A New Dataset for Interactive Digital Narrative Extractive Text Summarisation
Summarizing Interactive Digital Narratives (IDN) presents some unique challenges to existing text summarization models especially around capturing interactive elements in addition to important plot points. In this paper,…
Extractive Text SummarizationText SummarizationePiC: Employing Proverbs in Context as a Benchmark for Abstract Language Understanding
While large language models have shown exciting progress on several NLP benchmarks, evaluating their ability for complex analogical reasoning remains under-explored. Here, we introduce a high-quality crowdsourced dataset…
Bayesian Detector Combination for Object Detection with Crowdsourced Annotations
Acquiring fine-grained object detection annotations in unconstrained images is time-consuming, expensive, and prone to noise, especially in crowdsourcing scenarios. Most prior object detection methods assume accurate ann…
Objectobject-detectionObject Detection