Adversarial Training for Commonsense Inference
We propose an AdversariaL training algorithm for commonsense InferenCE (ALICE). We apply small perturbations to word embeddings and minimize the resultant adversarial risk to regularize the model. We exploit a novel combination of two different approaches to estimate these perturbations: 1) using the true label and 2) using the model prediction. Without relying on any human-crafted features, knowledge bases, or additional datasets other than the target datasets, our model boosts the fine-tuning performance of RoBERTa, achieving competitive results on multiple reading comprehension datasets that require commonsense inference.
Code (1)
Tasks
Reading ComprehensionWord EmbeddingsMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
SWAG: A Large-Scale Adversarial Dataset for Grounded Commonsense Inference
Given a partial description like "she opened the hood of the car," humans can reason about the situation and anticipate what might come next ("then, she examined the engine"). In this paper, we introduce the task of grou…
Common Sense ReasoningMultiple-choiceNatural Language InferenceQuestion AnsweringAdversarial Transformer Language Models for Contextual Commonsense Inference
Contextualized or discourse aware commonsense inference is the task of generating coherent commonsense assertions (i.e., facts) from a given story, and a particular sentence from that story. Some problems with the task a…
Knowledge GraphsLanguage ModelingLanguage ModellingSentenceCIKQA: Learning Commonsense Inference with a Unified Knowledge-in-the-loop QA Paradigm
Recently, the community has achieved substantial progress on many commonsense reasoning benchmarks. However, it is still unclear what is learned from the training process: the knowledge, inference capability, or both? We…
Question AnsweringTask 2HellaSwag: Can a Machine Really Finish Your Sentence?
Recent work by Zellers et al. (2018) introduced a new task of commonsense natural language inference: given an event description such as "A woman sits at a piano," a machine must select the most likely followup: "She set…
HellaSwagNatural Language InferenceSentenceSentence CompletionNot Blind but Silenced: Rebalancing Vision and Language via Adversarial Counter-Commonsense Equilibrium
During MLLM decoding, attention often abnormally concentrates on irrelevant image tokens. While existing research dismisses this as invalid noise and forcibly redirects attention to compel focusing on key image informati…