Self-training with Few-shot Rationalization
While pre-trained language models have obtained state-of-the-art performance for several natural language understanding tasks, they are quite opaque in terms of their decision-making process. While some recent works focus on rationalizing neural predictions by highlighting salient concepts in the text as justifications or rationales, they rely on thousands of labeled training examples for both task labels as well as annotated rationales for every instance. Such extensive large-scale annotations are infeasible to obtain for many tasks. To this end, we develop a multi-task teacher-student framework based on self-training pre-trained language models with limited task-specific labels and rationales and judicious sample selection to learn from informative pseudo-labeled examples. We study several characteristics of what constitutes a good rationale and demonstrate that the neural model performance can be significantly improved by making it aware of its rationalized predictions, particularly in low-resource settings. Extensive experiments in several benchmark datasets demonstrate the effectiveness of our approach.
Code (0)
등록된 구현이 없습니다.
Tasks
Decision MakingNatural Language UnderstandingSimilar Papers 제목 키워드 기반
Few-Shot Self-Rationalization with Natural Language Prompts
Self-rationalization models that predict task labels and generate free-text elaborations for their predictions could enable more intuitive interaction with NLP systems. These models are, however, currently trained with a…
Few-Shot Self-Rationalization with Natural Language Prompts
Self-rationalization models that predict task labels and generate free-text elaborations for their predictions could enable more intuitive interaction with NLP systems. These models are, however, currently trained with a…
ZARA: Improving Few-Shot Self-Rationalization for Small Language Models
Language models (LMs) that jointly generate end-task answers as well as free-text rationales are known as self-rationalization models. Recent works demonstrate great performance gain for self-rationalization by few-shot …
Natural Language InferenceTake It Easy: Label-Adaptive Self-Rationalization for Fact Verification and Explanation Generation
Computational methods to aid journalists in the task often require adapting a model to specific domains and generating explanations. However, most automated fact-checking methods rely on three-class datasets, which do no…
Explanation GenerationFact CheckingFact VerificationMisinformation+2Zero-Shot Rationalization by Multi-Task Transfer Learning from Question Answering
Extracting rationales can help human understand which information the model utilizes and how it makes the prediction towards better interpretability. However, annotating rationales requires much effort and only few datas…
Multi-Task LearningQuestion AnsweringTransfer Learning