Relation Regularized Scene Graph Generation
Scene graph generation (SGG) is built on top of detected objects to predict object pairwise visual relations for describing the image content abstraction. Existing works have revealed that if the links between objects are given as prior knowledge, the performance of SGG is significantly improved. Inspired by this observation, in this article, we propose a relation regularized network (R2-Net), which can predict whether there is a relationship between two objects and encode this relation into object feature refinement and better SGG. Specifically, we first construct an affinity matrix among detected objects to represent the probability of a relationship between two objects. Graph convolution networks (GCNs) over this relation affinity matrix are then used as object encoders, producing relation-regularized representations of objects. With these relation-regularized features, our R2-Net can effectively refine object labels and generate scene graphs. Extensive experiments are conducted on the visual genome dataset for three SGG tasks (i.e., predicate classification, scene graph classification, and scene graph detection), demonstrating the effectiveness of our proposed method. Ablation studies also verify the key roles of our proposed components in performance improvement.
Code (0)
등록된 구현이 없습니다.
Tasks
Graph ClassificationGraph GenerationObjectPredicate ClassificationRelationScene Graph ClassificationScene Graph DetectionScene Graph GenerationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
RU-Net: Regularized Unrolling Network for Scene Graph Generation
Scene graph generation (SGG) aims to detect objects and predict the relationships between each pair of objects. Existing SGG methods usually suffer from several issues, including 1) ambiguous object representations, as g…
DenoisingDiversityGraph GenerationGraph Neural Network+2Graphmax for Text Generation
In text generation, a large language model (LM) makes a choice of each new word based only on the former selection of its context using the softmax function. Nevertheless, the link statistics information of concurrent wo…
Language ModellingLarge Language ModelMachine TranslationText GenerationBCTR: Bidirectional Conditioning Transformer for Scene Graph Generation
Scene Graph Generation (SGG) remains a challenging task due to its compositional property. Previous approaches improve prediction efficiency through end-to-end learning. However, these methods exhibit limited performance…
Graph GenerationScene Graph GenerationHeuristics for Image Generation from Scene Graphs
Generating realistic images from scene graphs requires neural networks to be able to reason about object relationships and compositionality. Learning a sufficiently rich representation to facilitate this reasoning is cha…
Image GenerationImage Generation from Scene GraphsRelationHyperGLM: HyperGraph for Video Scene Graph Generation and Anticipation
Multimodal LLMs have advanced vision-language tasks but still struggle with understanding video scenes. To bridge this gap, Video Scene Graph Generation (VidSGG) has emerged to capture multi-object relationships across v…
Graph GenerationQuestion AnsweringScene Graph GenerationVideo Captioning+2