SciNLI: A Corpus for Natural Language Inference on Scientific Text
Existing Natural Language Inference (NLI) datasets, while being instrumental in the advancement of Natural Language Understanding (NLU) research, are not related to scientific text. In this paper, we introduce SciNLI, a large dataset for NLI that captures the formality in scientific text and contains 107,412 sentence pairs extracted from scholarly papers on NLP and computational linguistics. Given that the text used in scientific literature differs vastly from the text used in everyday language both in terms of vocabulary and sentence structure, our dataset is well suited to serve as a benchmark for the evaluation of scientific NLU models. Our experiments show that SciNLI is harder to classify than the existing NLI datasets. Our best performing model with XLNet achieves a Macro F1 score of only 78.18% and an accuracy of 78.23% showing that there is substantial room for improvement.
Code (1)
Tasks
Natural Language InferenceNatural Language UnderstandingSentenceMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
MSciNLI: A Diverse Benchmark for Scientific Natural Language Inference
The task of scientific Natural Language Inference (NLI) involves predicting the semantic relation between two sentences extracted from research articles. This task was recently proposed along with a new dataset called Sc…
ArticlesDiversityNatural Language InferenceSentence+1Co-training for Low Resource Scientific Natural Language Inference
Scientific Natural Language Inference (NLI) is the task of predicting the semantic relation between a pair of sentences extracted from research articles. The automatic annotation method based on distant supervision for t…
ArticlesNatural Language InferenceCSL: A Large-scale Chinese Scientific Literature Dataset
Scientific literature serves as a high-quality corpus, supporting a lot of Natural Language Processing (NLP) research. However, existing datasets are centered around the English language, which restricts the development …
text-classificationText ClassificationScientific Explanation and Natural Language: A Unified Epistemological-Linguistic Perspective for Explainable AI
A fundamental research goal for Explainable AI (XAI) is to build models that are capable of reasoning through the generation of natural language explanations. However, the methodologies to design and evaluate explanation…
Explainable Artificial Intelligence (XAI)PhilosophyA Multi-Layered Annotated Corpus of Scientific Papers
Scientific literature records the research process with a standardized structure and provides the clues to track the progress in a scientific field. Understanding its internal structure and content is of paramount import…
Sentence