From Representation to Reasoning: Towards both Evidence and Commonsense Reasoning for Video Question-Answering
Video understanding has achieved great success in representation learning, such as video caption, video object grounding, and video descriptive question-answer. However, current methods still struggle on video reasoning, including evidence reasoning and commonsense reasoning. To facilitate deeper video understanding towards video reasoning, we present the task of Causal-VidQA, which includes four types of questions ranging from scene description (description) to evidence reasoning (explanation) and commonsense reasoning (prediction and counterfactual). For commonsense reasoning, we set up a two-step solution by answering the question and providing a proper reason. Through extensive experiments on existing VideoQA methods, we find that the state-of-the-art methods are strong in descriptions but weak in reasoning. We hope that Causal-VidQA can guide the research of video understanding from representation learning to deeper reasoning. The dataset and related resources are available at \url{https://github.com/bcmi/Causal-VidQA.git}.
Code (1)
Tasks
counterfactualDescriptiveQuestion AnsweringRepresentation LearningVideo Question AnsweringVideo UnderstandingSimilar Papers 제목 키워드 기반
It's All in the Heads: Using Attention Heads as a Baseline for Cross-Lingual Transfer in Commonsense Reasoning
Commonsense reasoning is one of the key problems in natural language processing, but the relative scarcity of labeled data holds back the progress for languages other than English. Pretrained cross-lingual models are a s…
AllCross-Lingual TransferelBERto: Self-supervised Commonsense Learning for Question Answering
Commonsense question answering requires reasoning about everyday situations and causes and effects implicit in context. Typically, existing approaches first retrieve external evidence and then perform commonsense reasoni…
Question AnsweringRepresentation LearningRetrievalGraph-Based Reasoning over Heterogeneous External Knowledge for Commonsense Question Answering
Commonsense question answering aims to answer questions which require background knowledge that is not explicitly expressed in the question. The key challenge is how to obtain evidence from external knowledge and make pr…
Common Sense ReasoningGraph AttentionQuestion AnsweringRepresentation LearningDialectical language model evaluation: An initial appraisal of the commonsense spatial reasoning abilities of LLMs
Language models have become very popular recently and many claims have been made about their abilities, including for commonsense reasoning. Given the increasingly better results of current language models on previous st…
Language Model EvaluationLanguage ModelingLanguage ModellingSpatial ReasoningPrompting Contrastive Explanations for Commonsense Reasoning Tasks
Many commonsense reasoning NLP tasks involve choosing between one or more possible answers to a question or prompt based on knowledge that is often implicit. Large pretrained language models (PLMs) can achieve near-human…
Attribute