Causal Understanding For Video Question Answering
Video Question Answering is a challenging task, which requires the model to reason over multiple frames and understand the interaction between different objects to answer questions based on the context provided within the video, especially in datasets like NExT-QA (Xiao et al., 2021a) which emphasize on causal and temporal questions. Previous approaches leverage either sub-sampled information or causal intervention techniques along with complete video features to tackle the NExT-QA task. In this work we elicit the limitations of these approaches and propose solutions along four novel directions of improvements on theNExT-QA dataset. Our approaches attempts to compensate for the shortcomings in the previous works by systematically attacking each of these problems by smartly sampling frames, explicitly encoding actions and creating interventions that challenge the understanding of the model. Overall, for both single-frame (+6.3%) and complete-video (+1.1%) based approaches, we obtain the state-of-the-art results on NExT-QA dataset.
Code (0)
등록된 구현이 없습니다.
Tasks
Question AnsweringVideo Question AnsweringSimilar Papers 제목 키워드 기반
From Representation to Reasoning: Towards both Evidence and Commonsense Reasoning for Video Question-Answering
Video understanding has achieved great success in representation learning, such as video caption, video object grounding, and video descriptive question-answer. However, current methods still struggle on video reasoning,…
counterfactualDescriptiveQuestion AnsweringRepresentation Learning+2Equivariant and Invariant Grounding for Video Question Answering
Video Question Answering (VideoQA) is the task of answering the natural language questions about a video. Producing an answer requires understanding the interplay across visual scenes in video and linguistic semantics in…
Question AnsweringVideo Question AnsweringKARMA-MV: A Benchmark for Causal Question Answering on Music Videos
While significant progress has been made in Video Question Answering and cross-modal understanding, causal reasoning about how visual dynamics drive musical structure in music videos remains under-explored. We introduce …
Video Question AnsweringInvariant Grounding for Video Question Answering
Video Question Answering (VideoQA) is the task of answering questions about a video. At its core is understanding the alignments between visual scenes in video and linguistic semantics in question to yield the answer. In…
Question AnsweringVideo Question AnsweringMECD+: Unlocking Event-Level Causal Graph Discovery for Video Reasoning
Video causal reasoning aims to achieve a high-level understanding of videos from a causal perspective. However, it exhibits limitations in its scope, primarily executed in a question-answering paradigm and focusing on br…
Causal DiscoveryCausal InferencecounterfactualCounterfactual Inference+3