Vision Relation Transformer for Unbiased Scene Graph Generation
Recent years have seen a growing interest in Scene Graph Generation (SGG), a comprehensive visual scene understanding task that aims to predict entity relationships using a relation encoder-decoder pipeline stacked on top of an object encoder-decoder backbone. Unfortunately, current SGG methods suffer from an information loss regarding the entities local-level cues during the relation encoding process. To mitigate this, we introduce the Vision rElation TransfOrmer (VETO), consisting of a novel local-level entity relation encoder. We further observe that many existing SGG methods claim to be unbiased, but are still biased towards either head or tail classes. To overcome this bias, we introduce a Mutually Exclusive ExperT (MEET) learning strategy that captures important relation features without bias towards head or tail classes. Experimental results on the VG and GQA datasets demonstrate that VETO + MEET boosts the predictive performance by up to 47 percentage over the state of the art while being 10 times smaller.
Code (1)
Tasks
DecoderGraph GenerationRelationScene Graph GenerationScene UnderstandingUnbiased Scene Graph GenerationSimilar Papers 제목 키워드 기반
Unbiased Scene Graph Generation in Videos
The task of dynamic scene graph generation (SGG) from videos is complicated and challenging due to the inherent dynamics of a scene, temporal fluctuation of model predictions, and the long-tailed distribution of the visu…
Graph GenerationScene Graph GenerationUnbiased Scene Graph GenerationCogTree: Cognition Tree Loss for Unbiased Scene Graph Generation
Scene graphs are semantic abstraction of images that encourage visual understanding and reasoning. However, the performance of Scene Graph Generation (SGG) is unsatisfactory when faced with biased data in real-world scen…
Graph GenerationScene Graph GenerationUnbiased Scene Graph GenerationResistance Training using Prior Bias: toward Unbiased Scene Graph Generation
Scene Graph Generation (SGG) aims to build a structured representation of a scene using objects and pairwise relationships, which benefits downstream tasks. However, current SGG methods usually suffer from sub-optimal sc…
Graph GenerationScene Graph GenerationUnbiased Scene Graph GenerationFloCoDe: Unbiased Dynamic Scene Graph Generation with Temporal Consistency and Correlation Debiasing
Dynamic scene graph generation (SGG) from videos requires not only a comprehensive understanding of objects across scenes but also a method to capture the temporal motions and interactions with different objects. Moreove…
Graph GenerationScene Graph GenerationUnbiased Scene Graph GenerationIterative Scene Graph Generation with Generative Transformers
Scene graphs provide a rich, structured representation of a scene by encoding the entities (objects) and their spatial relationships in a graphical format. This representation has proven useful in several tasks, such as …
Graph GenerationLink Predictionobject-detectionObject Detection+3