Visual Social Relationship Recognition
Social relationships form the basis of social structure of humans. Developing computational models to understand social relationships from visual data is essential for building intelligent machines that can better interact with humans in a social environment. In this work, we study the problem of visual social relationship recognition in images. We propose a Dual-Glance model for social relationship recognition, where the first glance fixates at the person of interest and the second glance deploys attention mechanism to exploit contextual cues. To enable this study, we curated a large scale People in Social Context (PISC) dataset, which comprises of 23,311 images and 79,244 person pairs with annotated social relationships. Since visually identifying social relationship bears certain degree of uncertainty, we further propose an Adaptive Focal Loss to leverage the ambiguous annotations for more effective learning. We conduct extensive experiments to quantitatively and qualitatively demonstrate the efficacy of our proposed method, which yields state-of-the-art performance on social relationship recognition.
Code (0)
등록된 구현이 없습니다.
Tasks
Visual Social Relationship RecognitionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
From a Social Cognitive Perspective: Context-aware Visual Social Relationship Recognition
People's social relationships are often manifested through their surroundings, with certain objects or interactions acting as symbols for specific relationships, e.g., wedding rings, roses, hugs, or holding hands. This b…
DescriptiveVisual Social Relationship RecognitionAn End-to-End Network for Generating Social Relationship Graphs
Socially-intelligent agents are of growing interest in artificial intelligence. To this end, we need systems that can understand social relationships in diverse social contexts. Inferring the social context in a given vi…
AttributeGraph GenerationVisual Social Relationship RecognitionLove in Action: Gamifying Public Video Cameras for Fostering Social Relationships in Real World
In this paper, we create "Love in Action" (LIA), a body language-based social game utilizing video cameras installed in public spaces to enhance social relationships in real-world. In the game, participants assume dual r…
Action RecognitionAttributeHuman DetectionDual-Glance Model for Deciphering Social Relationships
Since the beginning of early civilizations, social relationships derived from each individual fundamentally form the basis of social structure in our daily life. In the computer vision literature, much progress has been …
modelobject-detectionObject DetectionScene Parsing+2Aiding Intra-Text Representations with Visual Context for Multimodal Named Entity Recognition
With massive explosion of social media such as Twitter and Instagram, people daily share billions of multimedia posts, containing images and text. Typically, text in these posts is short, informal and noisy, leading to a…
named-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)