Unbiased Video Scene Graph Generation via Visual and Semantic Dual Debiasing
Video Scene Graph Generation (VidSGG) aims to capture dynamic relationships among entities by sequentially analyzing video frames and integrating visual and semantic information. However, VidSGG is challenged by significant biases that skew predictions. To mitigate these biases, we propose a VIsual and Semantic Awareness (VISA) framework for unbiased VidSGG. VISA addresses visual bias through memory-enhanced temporal integration that enhances object representations and concurrently reduces semantic bias by iteratively integrating object features with comprehensive semantic information derived from triplet relationships. This visual-semantics dual debiasing approach results in more unbiased representations of complex scene dynamics. Extensive experiments demonstrate the effectiveness of our method, where VISA outperforms existing unbiased VidSGG approaches by a substantial margin (e.g., +13.1% improvement in mR@20 and mR@50 for the SGCLS task under Semi Constraint).
Code (0)
등록된 구현이 없습니다.
Tasks
Graph GenerationScene Graph GenerationTripletVideo scene graph generationSimilar Papers 제목 키워드 기반
Unbiased Scene Graph Generation in Videos
The task of dynamic scene graph generation (SGG) from videos is complicated and challenging due to the inherent dynamics of a scene, temporal fluctuation of model predictions, and the long-tailed distribution of the visu…
Graph GenerationScene Graph GenerationUnbiased Scene Graph GenerationFloCoDe: Unbiased Dynamic Scene Graph Generation with Temporal Consistency and Correlation Debiasing
Dynamic scene graph generation (SGG) from videos requires not only a comprehensive understanding of objects across scenes but also a method to capture the temporal motions and interactions with different objects. Moreove…
Graph GenerationScene Graph GenerationUnbiased Scene Graph GenerationTowards Unbiased and Robust Spatio-Temporal Scene Graph Generation and Anticipation
Spatio-Temporal Scene Graphs (STSGs) provide a concise and expressive representation of dynamic scenes by modelling objects and their evolving relationships over time. However, real-world visual relationships often exhib…
Graph GenerationScene Graph GenerationSpatio-temporal Scene GraphsVideo scene graph generationTriple Correlations-Guided Label Supplementation for Unbiased Video Scene Graph Generation
Video-based scene graph generation (VidSGG) is an approach that aims to represent video content in a dynamic graph by identifying visual entities and their relationships. Due to the inherently biased distribution and mis…
Graph GenerationMissing LabelsScene Graph GenerationVideo scene graph generationBipartite Graph Network with Adaptive Message Passing for Unbiased Scene Graph Generation
Scene graph generation is an important visual understanding task with a broad range of vision applications. Despite recent tremendous progress, it remains challenging due to the intrinsic long-tailed class distribution a…
Graph GenerationGraph Neural NetworkScene Graph GenerationUnbiased Scene Graph Generation