Learning To Generate Scene Graph from Head to Tail
Scene Graph Generation (SGG) represents objects and their interactions with a graph structure. Recently, many works are devoted to solving the imbalanced problem in SGG. However, underestimating the head predicates in the whole training process, they wreck the features of head predicates that provide general features for tail ones. Besides, assigning excessive attention to the tail predicates leads to semantic deviation. Based on this, we propose a novel SGG framework, learning to generate scene graphs from Head to Tail (SGG-HT), containing Curriculum Re-weight Mechanism (CRM) and Semantic Context Module (SCM). CRM learns head/easy samples firstly for robust features of head predicates and then gradually focuses on tail/hard ones. SCM is proposed to relieve semantic deviation by ensuring the semantic consistency between the generated scene graph and the ground truth in global and local representations. Experiments show that SGG-HT significantly alleviates the biased problem and chieves state-of-the-art performances on Visual Genome.
Code (0)
등록된 구현이 없습니다.
Tasks
Graph GenerationScene Graph GenerationSimilar Papers 제목 키워드 기반
Dual-branch Hybrid Learning Network for Unbiased Scene Graph Generation
The current studies of Scene Graph Generation (SGG) focus on solving the long-tailed problem for generating unbiased scene graphs. However, most de-biasing methods overemphasize the tail predicates and underestimate head…
Graph GenerationImage CaptioningRetrievalScene Graph Generation+2PRISM: Progressive Restoration for Scene Graph-based Image Manipulation
Scene graphs have emerged as accurate descriptive priors for image generation and manipulation tasks, however, their complexity and diversity of the shapes and relations of objects in data make it challenging to incorpor…
DenoisingDescriptiveDiversityImage Generation+1Head-Tail Cooperative Learning Network for Unbiased Scene Graph Generation
Scene Graph Generation (SGG) as a critical task in image understanding, facing the challenge of head-biased prediction caused by the long-tail distribution of predicates. However, current unbiased SGG methods can easily …
Graph GenerationPredictionScene Graph GenerationSelf-Supervised Learning+1Voxel Scene Graph for Intracranial Hemorrhage
Patients with Intracranial Hemorrhage (ICH) face a potentially life-threatening condition, and patient-centered individualized treatment remains challenging due to possible clinical complications. Deep-Learning-based met…
Decision MakingGraph Generationobject-detectionObject Detection+1Towards Unbiased and Robust Spatio-Temporal Scene Graph Generation and Anticipation
Spatio-Temporal Scene Graphs (STSGs) provide a concise and expressive representation of dynamic scenes by modelling objects and their evolving relationships over time. However, real-world visual relationships often exhib…
Graph GenerationScene Graph GenerationSpatio-temporal Scene GraphsVideo scene graph generation