Pay Attention to the Ending:Strong Neural Baselines for the ROC Story Cloze Task
We consider the ROC story cloze task (Mostafazadeh et al., 2016) and present several findings. We develop a model that uses hierarchical recurrent networks with attention to encode the sentences in the story and score candidate endings. By discarding the large training set and only training on the validation set, we achieve an accuracy of 74.7{\%}. Even when we discard the story plots (sentences before the ending) and only train to choose the better of two endings, we can still reach 72.5{\%}. We then analyze this {``}ending-only{''} task setting. We estimate human accuracy to be 78{\%} and find several types of clues that lead to this high accuracy, including those related to sentiment, negation, and general ending likelihood regardless of the story context.
Code (0)
등록된 구현이 없습니다.
Tasks
NegationOutlier DetectionSentiment AnalysisSimilar Papers 제목 키워드 기반
Story Cloze Ending Selection Baselines and Data Examination
This paper describes two supervised baseline systems for the Story Cloze Test Shared Task (Mostafazadeh et al., 2016a). We first build a classifier using features based on word embeddings and semantic similarity computat…
Cloze TestSemantic SimilaritySemantic Textual SimilarityWord EmbeddingsA Simple and Effective Approach to the Story Cloze Test
In the Story Cloze Test, a system is presented with a 4-sentence prompt to a story, and must determine which one of two potential endings is the 'right' ending to the story. Previous work has shown that ignoring the trai…
Cloze TestFeature EngineeringQuestion AnsweringSentenceLSDSem 2017 Shared Task: The Story Cloze Test
The LSDSem{'}17 shared task is the Story Cloze Test, a new evaluation for story understanding and script learning. This test provides a system with a four-sentence story and two possible endings, and the system must choo…
Cloze TestReading ComprehensionSentenceThe Effect of Different Writing Tasks on Linguistic Style: A Case Study of the ROC Story Cloze Task
A writer's style depends not just on personal traits but also on her intent and mental state. In this paper, we show how variants of the same writing task can lead to measurable differences in writing style. We present a…
Language ModelingLanguage ModellingNarrative Modeling with Memory Chains and Semantic Supervision
Story comprehension requires a deep semantic understanding of the narrative, making it a challenging task. Inspired by previous studies on ROC Story Cloze Test, we propose a novel method, tracking various semantic aspect…
Cloze Test