Probing the Natural Language Inference Task with Automated Reasoning Tools
The Natural Language Inference (NLI) task is an important task in modern NLP, as it asks a broad question to which many other tasks may be reducible: Given a pair of sentences, does the first entail the second? Although the state-of-the-art on current benchmark datasets for NLI are deep learning-based, it is worthwhile to use other techniques to examine the logical structure of the NLI task. We do so by testing how well a machine-oriented controlled natural language (Attempto Controlled English) can be used to parse NLI sentences, and how well automated theorem provers can reason over the resulting formulae. To improve performance, we develop a set of syntactic and semantic transformation rules. We report their performance, and discuss implications for NLI and logic-based NLP.
Code (1)
Tasks
Natural Language InferenceSimilar Papers 제목 키워드 기반
Probing Linguistic Information For Logical Inference In Pre-trained Language Models
Progress in pre-trained language models has led to a surge of impressive results on downstream tasks for natural language understanding. Recent work on probing pre-trained language models uncovered a wide range of lingui…
Language ModelingLanguage ModellingNatural Language UnderstandingInterventional Probing in High Dimensions: An NLI Case Study
Probing strategies have been shown to detect the presence of various linguistic features in large language models; in particular, semantic features intermediate to the "natural logic" fragment of the Natural Language Inf…
Natural Language InferenceVocal Bursts Intensity PredictionLINSPECTOR WEB: A Multilingual Probing Suite for Word Representations
We present LINSPECTOR WEB, an open source multilingual inspector to analyze word representations. Our system provides researchers working in low-resource settings with an easily accessible web based probing tool to gain …
Dependency Parsingnamed-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)+2Probing Multilingual Sentence Representations With X-Probe
This paper extends the task of probing sentence representations for linguistic insight in a multilingual domain. In doing so, we make two contributions: first, we provide datasets for multilingual probing, derived from W…
Natural Language InferenceSentenceProbing What Different NLP Tasks Teach Machines about Function Word Comprehension
We introduce a set of nine challenge tasks that test for the understanding of function words. These tasks are created by structurally mutating sentences from existing datasets to target the comprehension of specific type…
CCG SupertaggingLanguage ModelingLanguage ModellingNatural Language Inference+2