paper-with-me

홈 › Papers

QUASER: Question Answering with Scalable Extractive Rationalization

2021-05-16 · ACL ARR May 2021 5 · Anonymous

Designing NLP models that produce predictions by first extracting a set of relevant input sentences (i.e., rationales), is gaining importance as a means to improving model interpretability and to producing supporting evidence for users. Current unsupervised approaches are trained to extract rationales that maximize prediction accuracy, which is invariably obtained by exploiting spurious correlations in datasets, and leads to unconvincing rationales. In this paper, we introduce unsupervised generative models to extract dual-purpose rationales, which must not only be able to support a subsequent answer prediction, but also support a reproduction of the input query. We show that such models can produce more meaningful rationales, that are less influenced by dataset artifacts, and as a result, also achieve the state-of-the-art on rationale extraction metrics on four datasets from the ERASER benchmark, significantly improving upon previous unsupervised methods.

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Tasks

Question Answering

Similar Papers 제목 키워드 기반

Measuring Association Between Labels and Free-Text Rationales

2020-10-24 · EMNLP 2021 11 · Sarah Wiegreffe, Ana Marasović, Noah A. Smith

In interpretable NLP, we require faithful rationales that reflect the model's decision-making process for an explained instance. While prior work focuses on extractive rationales (a subset of the input words), we investi…

Decision MakingFeature ImportanceNatural Language InferenceQuestion Answering

Zero-Shot Rationalization by Multi-Task Transfer Learning from Question Answering

2020-11-01 · Findings of the Association for Computational Linguistics 2020 · Po-Nien Kung, Tse-Hsuan Yang, Yi-Cheng Chen, Sheng-Siang Yin 외

Extracting rationales can help human understand which information the model utilizes and how it makes the prediction towards better interpretability. However, annotating rationales requires much effort and only few datas…

Multi-Task LearningQuestion AnsweringTransfer Learning

Plausible Extractive Rationalization through Semi-Supervised Entailment Signal

2024-02-13 · Wei Jie Yeo, Ranjan Satapathy, Erik Cambria

The increasing use of complex and opaque black box models requires the adoption of interpretable measures, one such option is extractive rationalizing models, which serve as a more interpretable alternative. These models…

Natural Language InferenceQuestion Answering

On scalable oversight with weak LLMs judging strong LLMs

2024-07-05 · Zachary Kenton, Noah Y. Siegel, János Kramár, Jonah Brown-Cohen 외

Scalable oversight protocols aim to enable humans to accurately supervise superhuman AI. In this paper we study debate, where two AI's compete to convince a judge; consultancy, where a single AI tries to convince a judge…

Multimodal ReasoningQuestion Answering

MemSum-DQA: Adapting An Efficient Long Document Extractive Summarizer for Document Question Answering

2023-10-10 · Nianlong Gu, Yingqiang Gao, Richard H. R. Hahnloser

We introduce MemSum-DQA, an efficient system for document question answering (DQA) that leverages MemSum, a long document extractive summarizer. By prefixing each text block in the parsed document with the provided quest…

Extractive SummarizationQuestion Answering