Demo2Vec: Reasoning Object Affordances From Online Videos
Watching expert demonstrations is an important way for humans and robots to reason about affordances of unseen objects. In this paper, we consider the problem of reasoning object affordances through the feature embedding of demonstration videos. We design the Demo2Vec model which learns to extract embedded vectors of demonstration videos and predicts the interaction region and the action label on a target image of the same object. We introduce the Online Product Review dataset for Affordance (OPRA) by collecting and labeling diverse YouTube product review videos. Our Demo2Vec model outperforms various recurrent neural network baselines on the collected dataset.
Code (0)
등록된 구현이 없습니다.
Tasks
ObjectVideo-to-image Affordance GroundingSimilar Papers 제목 키워드 기반
What Objects Enable, Not What They Are: Functional Latent Spaces for Affordance Reasoning
Existing robot planning systems rely on appearance-based reasoning, where visual observations are encoded into latent spaces organized around object appearances (e.g., recognizing a "cart" based on how it looks). However…
Learning Human Activities and Object Affordances from RGB-D Videos
Understanding human activities and object affordances are two very important skills, especially for personal robots which operate in human environments. In this work, we consider the problem of extracting a descriptive l…
DescriptiveObjectSkeleton Based Action RecognitionTEXT2AFFORD: Probing Object Affordance Prediction abilities of Language Models solely from Text
We investigate the knowledge of object affordances in pre-trained language models (LMs) and pre-trained Vision-Language models (VLMs). A growing body of literature shows that PTLMs fail inconsistently and non-intuitively…
ObjectGLOVER++: Unleashing the Potential of Affordance Learning from Human Behaviors for Robotic Manipulation
Learning manipulation skills from human demonstration videos offers a promising path toward generalizable and interpretable robotic intelligence-particularly through the lens of actionable affordances. However, transferr…
BenchmarkingObject-agnostic Affordance Categorization via Unsupervised Learning of Graph Embeddings
Acquiring knowledge about object interactions and affordances can facilitate scene understanding and human-robot collaboration tasks. As humans tend to use objects in many different ways depending on the scene and the ob…
ObjectScene Understanding