Bridge to Answer: Structure-aware Graph Interaction Network for Video Question Answering
This paper presents a novel method, termed Bridge to Answer, to infer correct answers for questions about a given video by leveraging adequate graph interactions of heterogeneous crossmodal graphs. To realize this, we learn question conditioned visual graphs by exploiting the relation between video and question to enable each visual node using question-to-visual interactions to encompass both visual and linguistic cues. In addition, we propose bridged visual-to-visual interactions to incorporate two complementary visual information on appearance and motion by placing the question graph as an intermediate bridge. This bridged architecture allows reliable message passing through compositional semantics of the question to generate an appropriate answer. As a result, our method can learn the question conditioned visual representations attributed to appearance and motion that show powerful capability for video question answering. Extensive experiments prove that the proposed method provides effective and superior performance than state-of-the-art methods on several benchmarks.
Code (0)
등록된 구현이 없습니다.
Tasks
Question AnsweringVideo Question AnsweringSimilar Papers 제목 키워드 기반
G-SAP: Graph-based Structure-Aware Prompt Learning over Heterogeneous Knowledge for Commonsense Reasoning
Commonsense question answering has demonstrated considerable potential across various applications like assistants and social robots. Although fully fine-tuned pre-trained Language Models(LM) have achieved remarkable per…
Graph Neural NetworkKnowledge GraphsPrompt LearningQuestion AnsweringBridging the KB-Text Gap: Leveraging Structured Knowledge-aware Pre-training for KBQA
Knowledge Base Question Answering (KBQA) aims to answer natural language questions with factual information such as entities and relations in KBs. However, traditional Pre-trained Language Models (PLMs) are directly pre-…
Knowledge Base Question AnsweringQuestion AnsweringRetrievalCORN: Co-Reasoning Network for Commonsense Question Answering
Commonsense question answering (QA) requires machines to utilize the QA content and external commonsense knowledge graph (KG) for reasoning when answering questions. Existing work uses two independent modules to model th…
Question AnsweringContextualized Knowledge-aware Attentive Neural Network: Enhancing Answer Selection with Knowledge
Answer selection, which is involved in many natural language processing applications such as dialog systems and question answering (QA), is an important yet challenging task in practice, since conventional methods typica…
Answer SelectionQuestion AnsweringRepresentation LearningSentenceConflict-Resilient Multi-Agent Reasoning via Signed Graph Modeling
LLM-based multi-agent systems (MAS) have demonstrated strong reasoning and decision-making capabilities that consistently surpass those of single LLM agents. However, their performance often suffers from naive aggregatio…