Reasoning About Human-Object Interactions Through Dual Attention Networks
Objects are entities we act upon, where the functionality of an object is determined by how we interact with it. In this work we propose a Dual Attention Network model which reasons about human-object interactions. The dual-attentional framework weights the important features for objects and actions respectively. As a result, the recognition of objects and actions mutually benefit each other. The proposed model shows competitive classification performance on the human-object interaction dataset Something-Something. Besides, it can perform weak spatiotemporal localization and affordance segmentation, despite being trained only with video-level labels. The model not only finds when an action is happening and which object is being manipulated, but also identifies which part of the object is being interacted with. Project page: \url{https://dual-attention-network.github.io/}.
Code (0)
등록된 구현이 없습니다.
Tasks
Human-Object Interaction DetectionObjectSimilar Papers 제목 키워드 기반
Reconstructing Action-Conditioned Human-Object Interactions Using Commonsense Knowledge Priors
We present a method for inferring diverse 3D models of human-object interactions from images. Reasoning about how humans interact with objects in complex scenes from a single 2D image is a challenging task given ambiguit…
Human-Object Interaction DetectionObjectCRAFT: A Benchmark for Causal Reasoning About Forces and inTeractions
Humans are able to perceive, understand and reason about causal events. Developing models with similar physical and causal understanding capabilities is a long-standing goal of artificial intelligence. As a step towards …
counterfactualDescriptiveQuestion AnsweringVideo Question Answering+1CRAFT: A Benchmark for Causal Reasoning About Forces and inTeractions
Humans are able to perceive, understand and reason about causal events. Developing models with similar physical and causal understanding capabilities is a long-standing goal of artificial intelligence. As a step towards …
counterfactualDescriptiveQuestion AnsweringVideo Question AnsweringSlotPi: Physics-informed Object-centric Reasoning Models
Understanding and reasoning about dynamics governed by physical laws through visual observation, akin to human capabilities in the real world, poses significant challenges. Currently, object-centric dynamic simulation me…
ObjectQuestion AnsweringVisual Question AnsweringVisual Question Answering (VQA)Object Level Visual Reasoning in Videos
Human activity recognition is typically addressed by detecting key concepts like global and local motion, features related to object classes present in the scene, as well as features related to the global context. The ne…
Activity RecognitionHuman Activity RecognitionObjectobject-detection+2