Using Natural Language Relations between Answer Choices for Machine Comprehension
When evaluating an answer choice for Reading Comprehension task, other answer choices available for the question and the answers of related questions about the same paragraph often provide valuable information. In this paper, we propose a method to leverage the natural language relations between the answer choices, such as entailment and contradiction, to improve the performance of machine comprehension. We use a stand-alone question answering (QA) system to perform QA task and a Natural Language Inference (NLI) system to identify the relations between the choice pairs. Then we perform inference using an Integer Linear Programming (ILP)-based relational framework to re-evaluate the decisions made by the standalone QA system in light of the relations identified by the NLI system. We also propose a multitask learning model that learns both the tasks jointly.
Code (1)
Tasks
Natural Language InferenceQuestion AnsweringReading ComprehensionSimilar Papers 제목 키워드 기반
GraghVQA: Language-Guided Graph Neural Networks for Graph-based Visual Question Answering
Images are more than a collection of objects or attributes -- they represent a web of relationships among interconnected objects. Scene Graph has emerged as a new modality for a structured graphical representation of ima…
Graph Neural NetworkGraph Question AnsweringQuestion AnsweringVisual Question Answering+1What's Missing: A Knowledge Gap Guided Approach for Multi-hop Question Answering
Multi-hop textual question answering requires combining information from multiple sentences. We focus on a natural setting where, unlike typical reading comprehension, only partial information is provided with each quest…
Multi-hop Question AnsweringQuestion AnsweringReading ComprehensionANPMI: Assessing the True Comprehension Capabilities of LLMs for Multiple Choice Questions
Multiple-choice benchmarks, consisting of various prompts and choices, are among the most widely used methods to assess a language model's natural language understanding capability. Given a specific prompt, we typically …
Language ModelingLanguage ModellingMultiple-choiceNatural Language UnderstandingDo LLMs Make Mistakes Like Students? Exploring Natural Alignment between Language Models and Human Error Patterns
Large Language Models (LLMs) have demonstrated remarkable capabilities in various educational tasks, yet their alignment with human learning patterns, particularly in predicting which incorrect options students are most …
Distractor GenerationMultiple-choiceMultilingual textual data: an approach through multiple factor analysis
This paper focuses on the analysis of open-ended questions answered in different languages. Closed-ended questions, called contextual variables, are asked to all respondents in order to understand the relationships betwe…