Spatial-Temporal Human-Object Interaction Detection
In this paper, we propose a new instance-level human-object interaction detection task on videos called ST-HOID, which aims to distinguish fine-grained human-object interactions (HOIs) and the trajectories of subjects and objects. It is motivated by the fact that HOI is crucial for human-centric video content understanding. To solve ST-HOID, we propose a novel method consisting of an object trajectory detection module and an interaction reasoning module. Furthermore, we construct the first dataset named VidOR-HOID for ST-HOID evaluation, which contains 10,831 spatial-temporal HOI instances. We conduct extensive experiments to evaluate the effectiveness of our method. The experimental results demonstrate that our method outperforms the baselines generated by the state-of-the-art methods of image human-object interaction detection, video visual relation detection and video human-object interaction recognition.
Code (0)
등록된 구현이 없습니다.
Tasks
Human-Object Interaction DetectionVideo Visual Relation DetectionSimilar Papers 제목 키워드 기반
Spatial Parsing and Dynamic Temporal Pooling networks for Human-Object Interaction detection
The key of Human-Object Interaction(HOI) recognition is to infer the relationship between human and objects. Recently, the image's Human-Object Interaction(HOI) detection has made significant progress. However, there is …
Human-Object Interaction DetectionObjectvalidST-HOI: A Spatial-Temporal Baseline for Human-Object Interaction Detection in Videos
Detecting human-object interactions (HOI) is an important step toward a comprehensive visual understanding of machines. While detecting non-temporal HOIs (e.g., sitting on a chair) from static images is feasible, it is u…
Action DetectionHuman-Object Interaction AnticipationHuman-Object Interaction DetectionSpatio-Temporal Action LocalizationEgoInteract: Synthetic Egocentric Videos Generation for Interaction Understanding and Anticipation
Collecting large-scale egocentric video datasets with dense spatial and temporal annotations is costly, slow, and often constrained by environmental biases, privacy constraints, and limited coverage of interaction patter…
Active Object DetectionAction SegmentationVideo GenerationVideo-based Human-Object Interaction Detection from Tubelet Tokens
We present a novel vision Transformer, named TUTOR, which is able to learn tubelet tokens, served as highly-abstracted spatiotemporal representations, for video-based human-object interaction (V-HOI) detection. The tubel…
Human-Object Interaction DetectionLIGHTEN: Learning Interactions with Graph and Hierarchical TEmporal Networks for HOI in videos
Analyzing the interactions between humans and objects from a video includes identification of the relationships between humans and the objects present in the video. It can be thought of as a specialized version of Visual…
Human-Object Interaction DetectionRelationship DetectionVideo Visual Relation DetectionVisual Relationship Detection