Cracking the Contextual Commonsense Code: Understanding Commonsense Reasoning Aptitude of Deep Contextual Representations
Pretrained deep contextual representations have advanced the state-of-the-art on various commonsense NLP tasks, but we lack a concrete understanding of the capability of these models. Thus, we investigate and challenge several aspects of BERT's commonsense representation abilities. First, we probe BERT's ability to classify various object attributes, demonstrating that BERT shows a strong ability in encoding various commonsense features in its embedding space, but is still deficient in many areas. Next, we show that, by augmenting BERT's pretraining data with additional data related to the deficient attributes, we are able to improve performance on a downstream commonsense reasoning task while using a minimal amount of data. Finally, we develop a method of fine-tuning knowledge graphs embeddings alongside BERT and show the continued importance of explicit knowledge graphs.
Code (0)
등록된 구현이 없습니다.
Tasks
Knowledge GraphsMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Commonsense Reasoning in Computer Vision: Foundations, Recent Advancements, and Future Directions
Commonsense reasoning in computer vision encompasses integrating visual data and contextual knowledge, crucial for enhancing AI's understanding of everyday scenarios. This understanding not only improves machine learning…
Object RecognitionKnowledge GraphsVLC-BERT: Visual Question Answering with Contextualized Commonsense Knowledge
There has been a growing interest in solving Visual Question Answering (VQA) tasks that require the model to reason beyond the content present in the image. In this work, we focus on questions that require commonsense re…
Question AnsweringVisual Question AnsweringVisual Question Answering (VQA)ComFact: A Benchmark for Linking Contextual Commonsense Knowledge
Understanding rich narratives, such as dialogues and stories, often requires natural language processing systems to access relevant knowledge from commonsense knowledge graphs. However, these systems typically retrieve f…
Knowledge GraphsResponse GenerationRetrievalCommonsenseVIS: Visualizing and Understanding Commonsense Reasoning Capabilities of Natural Language Models
Recently, large pretrained language models have achieved compelling performance on commonsense benchmarks. Nevertheless, it is unclear what commonsense knowledge the models learn and whether they solely exploit spurious …
Question AnsweringRelational ReasoningDiffuCOMET: Contextual Commonsense Knowledge Diffusion
Inferring contextually-relevant and diverse commonsense to understand narratives remains challenging for knowledge models. In this work, we develop a series of knowledge models, DiffuCOMET, that leverage diffusion to lea…
Diversity