Understanding the Role of Scene Graphs in Visual Question Answering
Visual Question Answering (VQA) is of tremendous interest to the research community with important applications such as aiding visually impaired users and image-based search. In this work, we explore the use of scene graphs for solving the VQA task. We conduct experiments on the GQA dataset which presents a challenging set of questions requiring counting, compositionality and advanced reasoning capability, and provides scene graphs for a large number of images. We adopt image + question architectures for use with scene graphs, evaluate various scene graph generation techniques for unseen images, propose a training curriculum to leverage human-annotated and auto-generated scene graphs, and build late fusion architectures to learn from multiple image representations. We present a multi-faceted study into the use of scene graphs for VQA, making this work the first of its kind.
Code (0)
등록된 구현이 없습니다.
Tasks
Graph GenerationQuestion AnsweringScene Graph GenerationVisual Question AnsweringVisual Question Answering (VQA)Similar Papers 제목 키워드 기반
SGEITL: Scene Graph Enhanced Image-Text Learning for Visual Commonsense Reasoning
Answering complex questions about images is an ambitious goal for machine intelligence, which requires a joint understanding of images, text, and commonsense knowledge, as well as a strong reasoning ability. Recently, mu…
Visual Commonsense ReasoningExpressive Scene Graph Generation Using Commonsense Knowledge Infusion for Visual Understanding and Reasoning
Scene graph generation aims to capture the semantic elements in images by modelling objects and their relationships in a structured manner, which are essential for visual understanding and reasoning tasks including image…
Common Sense ReasoningGraph GenerationImage CaptioningImage Generation+9A Comprehensive Survey of Scene Graphs: Generation and Application
Scene graph is a structured representation of a scene that can clearly express the objects, attributes, and relationships between objects in the scene. As computer vision technology continues to develop, people are no lo…
Image CaptioningQuestion AnsweringRelationship DetectionRetrieval+5Graphhopper: Multi-Hop Scene Graph Reasoning for Visual Question Answering
Visual Question Answering (VQA) is concerned with answering free-form questions about an image. Since it requires a deep semantic and linguistic understanding of the question and the ability to associate it with various …
NavigateQuestion AnsweringVisual Question AnsweringVisual Question Answering (VQA)Visual Relationship Detection using Scene Graphs: A Survey
Understanding a scene by decoding the visual relationships depicted in an image has been a long studied problem. While the recent advances in deep learning and the usage of deep neural networks have achieved near human a…
Graph GenerationImage GenerationImage RetrievalObject Recognition+8