On the Evaluation of Answer-Agnostic Paragraph-level Multi-Question Generation

2022-03-09 · Jishnu Ray Chowdhury, Debanjan Mahata, Cornelia Caragea

We study the task of predicting a set of salient questions from a given paragraph without any prior knowledge of the precise answer. We make two main contributions. First, we propose a new method to evaluate a set of predicted questions against the set of references by using the Hungarian algorithm to assign predicted questions to references before scoring the assigned pairs. We show that our proposed evaluation strategy has better theoretical and practical properties compared to prior methods because it can properly account for the coverage of references. Second, we compare different strategies to utilize a pre-trained seq2seq model to generate and select a set of questions related to a given paragraph. The code is available.

📄 PDF Abstract BibTeX arXiv:2203.04464

Code (1)

jrc1995/questiongenerationpub 공식 구현 pytorch

Tasks

Question Generation Question-Generation

Methods 이 논문이 사용한 방법론

Sigmoid Activation 설명 없음

Tanh Activation 설명 없음

LSTM An LSTM is a type of recurrent neural network that addresses the vanishing gradient problem in vanilla…

Seq2Seq Seq2Seq, or Sequence To Sequence, is a model used in sequence prediction tasks, such as language modelling and machine translation. The idea is to use one…

CQA-Eval: Designing Reliable Evaluations of Multi-paragraph Clinical QA under Resource Constraints

2025-10-12 · Federica Bologna, Tiffany Pan, Matthew Wilkens, Yue Guo 외 arxiv

Evaluating multi-paragraph clinical question answering (QA) systems is resource-intensive and challenging: accurate judgments require medical expertise and achieving consistent human judgments over multi-paragraph text i…

Question Answering

Multi-step Retriever-Reader Interaction for Scalable Open-domain Question Answering

2019-05-14 · ICLR 2019 5 · Rajarshi Das, Shehzaad Dhuliawala, Manzil Zaheer, Andrew McCallum

This paper introduces a new framework for open-domain question answering in which the retriever and the reader iteratively interact with each other. The framework is agnostic to the architecture of the machine reading mo…

Open-Domain Question Answering Question Answering Reading Comprehension TriviaQA

Vocabulary Matters: A Simple yet Effective Approach to Paragraph-level Question Generation

2020-12-01 · Asian Chapter of the Association for Computational Linguistics 2020 · Vishwajeet Kumar, Manish Joshi, Ganesh Ramakrishnan, Yuan-Fang Li

Question generation (QG) has recently attracted considerable attention. Most of the current neural models take as input only one or two sentences, and perform poorly when multiple sentences or complete paragraphs are giv…

Question Generation Question-Generation

Legal Question Answering using Ranking SVM and Deep Convolutional Neural Network

2017-03-16 · Phong-Khac Do, Huy-Tien Nguyen, Chien-Xuan Tran, Minh-Tien Nguyen 외

This paper presents a study of employing Ranking SVM and Convolutional Neural Network for two missions: legal information retrieval and question answering in the Competition on Legal Information Extraction/Entailment. Fo…

Information Retrieval Question Answering Retrieval

Multi-Hop Paragraph Retrieval for Open-Domain Question Answering

2019-06-15 · ACL 2019 7 · Yair Feldman, Ran El-Yaniv

This paper is concerned with the task of multi-hop open-domain Question Answering (QA). This task is particularly challenging since it requires the simultaneous performance of textual reasoning and efficient searching. W…

Open-Domain Question Answering Question Answering Retrieval Sentence