Looking Beyond Short-Premise Natural Language Inference for Downstream Tasks
Natural Language Inference (NLI) has garnered significant attention in recent years; however, the promise of applying NLI breakthroughs to other downstream NLP tasks has remained unfulfilled. In this work, we use the multiple-choice reading comprehension (MCRC) and checking factual correctness of textual summarization (CFCS) tasks to investigate potential reasons for this. Our findings show that: (1) the relatively shorter length of premises in traditional NLI datasets is the primary challenge prohibiting usage in downstream applications (which do better with longer contexts); (2) this challenge can be addressed by automatically converting resource-rich reading comprehension datasets into longer-premise NLI datasets; and (3) models trained on the converted, longer-premise datasets outperform those trained using short-premise traditional NLI datasets on downstream tasks primarily due to the difference in premise lengths.
Code (0)
등록된 구현이 없습니다.
Tasks
Multiple-choiceNatural Language InferenceReading ComprehensionSimilar Papers 제목 키워드 기반
Looking Beyond Sentence-Level Natural Language Inference for Question Answering and Text Summarization
Natural Language Inference (NLI) has garnered significant attention in recent years; however, the promise of applying NLI breakthroughs to other downstream NLP tasks has remained unfulfilled. In this work, we use the mul…
Multiple-choiceNatural Language InferenceQuestion AnsweringReading Comprehension+2Looking Beyond Sentence-Level Natural Language Inference for Downstream Tasks
In recent years, the Natural Language Inference (NLI) task has garnered significant attention, with new datasets and models achieving near human-level performance on it. However, the full promise of NLI -- particularly t…
Natural Language InferenceQuestion AnsweringSentenceText SummarizationLawngNLI: a multigranular, long-premise NLI benchmark for evaluating models’ in-domain generalization from short to long contexts
Natural language inference has trended with NLP toward studying reasoning over long contexts, with several datasets moving beyond the sentence level. However, short-sequence models typically perform best despite their se…
Domain GeneralizationNatural Language InferenceRetrievalSentenceLawngNLI: A Long-Premise Benchmark for In-Domain Generalization from Short to Long Contexts and for Implication-Based Retrieval
Natural language inference has trended toward studying contexts beyond the sentence level. An important application area is law: past cases often do not foretell how they apply to new situations and implications must be …
Domain GeneralizationNatural Language InferenceRe-RankingRetrieval+1Annotation Artifacts in Natural Language Inference Data
Large-scale datasets for natural language inference are created by presenting crowd workers with a sentence (premise), and asking them to generate three new sentences (hypotheses) that it entails, contradicts, or is logi…
Natural Language InferenceNegationSentenceText Categorization