Teaching Pretrained Models with Commonsense Reasoning: A Preliminary KB-Based Approach
Recently, pretrained language models (e.g., BERT) have achieved great success on many downstream natural language understanding tasks and exhibit a certain level of commonsense reasoning ability. However, their performance on commonsense tasks is still far from that of humans. As a preliminary attempt, we propose a simple yet effective method to teach pretrained models with commonsense reasoning by leveraging the structured knowledge in ConceptNet, the largest commonsense knowledge base (KB). Specifically, the structured knowledge in KB allows us to construct various logical forms, and then generate multiple-choice questions requiring commonsense logical reasoning. Experimental results demonstrate that, when refined on these training examples, the pretrained models consistently improve their performance on tasks that require commonsense reasoning, especially in the few-shot learning setting. Besides, we also perform analysis to understand which logical relations are more relevant to commonsense reasoning.
Code (0)
등록된 구현이 없습니다.
Tasks
Few-Shot LearningLogical ReasoningMultiple-choiceNatural Language UnderstandingSimilar Papers 제목 키워드 기반
Commonsense Reasoning for Conversational AI: A Survey of the State of the Art
Large, transformer-based pretrained language models like BERT, GPT, and T5 have demonstrated a deep understanding of contextual semantics and language syntax. Their success has enabled significant advances in conversatio…
Commonsense Knowledge-Augmented Pretrained Language Models for Causal Reasoning Classification
Commonsense knowledge can be leveraged for identifying causal relations in text. In this work, we convert triples in ATOMIC2020, a wide coverage commonsense reasoning knowledge graph, to natural language text and continu…
ClassificationCommonsense Causal ReasoningLanguage ModelingLanguage ModellingKnowledge-Augmented Language Models for Cause-Effect Relation Classification
Previous studies have shown the efficacy of knowledge augmentation methods in pretrained language models. However, these methods behave differently across domains and downstream tasks. In this work, we investigate the au…
Cause-Effect Relation ClassificationClassificationCommonsense Causal ReasoningLanguage Modelling+2Incorporating Commonsense Knowledge Graph in Pretrained Models for Social Commonsense Tasks
Pretrained language models have excelled at many NLP tasks recently; however, their social intelligence is still unsatisfactory. To enable this, machines need to have a more general understanding of our complicated world…
Knowledge GraphsGo Beyond Plain Fine-tuning: Improving Pretrained Models for Social Commonsense
Pretrained language models have demonstrated outstanding performance in many NLP tasks recently. However, their social intelligence, which requires commonsense reasoning about the current situation and mental states of o…