Scene Graph Generation by Iterative Message Passing
Understanding a visual scene goes beyond recognizing individual objects in isolation. Relationships between objects also constitute rich semantic information about the scene. In this work, we explicitly model the objects and their relationships using scene graphs, a visually-grounded graphical structure of an image. We propose a novel end-to-end model that generates such structured scene representation from an input image. The model solves the scene graph inference problem using standard RNNs and learns to iteratively improves its predictions via message passing. Our joint inference model can take advantage of contextual cues to make better predictions on objects and their relationships. The experiments show that our model significantly outperforms previous methods for generating scene graphs using Visual Genome dataset and inferring support relations with NYU Depth v2 dataset.
Code (5)
Tasks
Graph GenerationPanoptic Scene Graph GenerationScene Graph GenerationSimilar Papers 제목 키워드 기반
SceneGraphNet: Neural Message Passing for 3D Indoor Scene Augmentation
In this paper we propose a neural message passing approach to augment an input 3D indoor scene with new objects matching their surroundings. Given an input, potentially incomplete, 3D scene and a query location, our meth…
3D Object RecognitionObject RecognitionScene GenerationUnbiased Heterogeneous Scene Graph Generation with Relation-aware Message Passing Neural Network
Recent scene graph generation (SGG) frameworks have focused on learning complex relationships among multiple objects in an image. Thanks to the nature of the message passing neural network (MPNN) that models high-order i…
Graph GenerationRelationRepresentation LearningScene Graph GenerationSelf-Supervised Relation Alignment for Scene Graph Generation
The goal of scene graph generation is to predict a graph from an input image, where nodes correspond to identified and localized objects and edges to their corresponding interaction predicates. Existing methods are train…
Graph GenerationRelationRelation PredictionScene Graph GenerationIterative Scene Graph Generation
The task of scene graph generation entails identifying object entities and their corresponding interaction predicates in a given image (or video). Due to the combinatorially large solution space, existing approaches to s…
Graph GenerationScene Graph GenerationConstrained Structure Learning for Scene Graph Generation
As a structured prediction task, scene graph generation aims to build a visually-grounded scene graph to explicitly model objects and their relationships in an input image. Currently, the mean field variational Bayesian …
Graph GenerationScene Graph GenerationStructured PredictionVariational Inference