Recognizing Manipulation Actions from State-Transformations
Manipulation actions transform objects from an initial state into a final state. In this paper, we report on the use of object state transitions as a mean for recognizing manipulation actions. Our method is inspired by the intuition that object states are visually more apparent than actions from a still frame and thus provide information that is complementary to spatio-temporal action recognition. We start by defining a state transition matrix that maps action labels into a pre-state and a post-state. From each keyframe, we learn appearance models of objects and their states. Manipulation actions can then be recognized from the state transition matrix. We report results on the EPIC kitchen action recognition challenge.
Code (1)
Tasks
Action RecognitionObjectSpatio-temporal Action RecognitionSimilar Papers 제목 키워드 기반
Problem Space Transformations for Out-of-Distribution Generalisation in Behavioural Cloning
The combination of behavioural cloning and neural networks has driven significant progress in robotic manipulation. As these algorithms may require a large number of demonstrations for each task of interest, they remain …
Behavioural cloningGlobal Symmetry and Orthogonal Transformations from Geometrical Moment $n$-tuples
Detecting symmetry is crucial for effective object grasping for several reasons. Recognizing symmetrical features or axes within an object helps in developing efficient grasp strategies, as grasping along these axes typi…
Planning from Point Clouds over Continuous Actions for Multi-object Rearrangement
Long-horizon planning for robot manipulation is a challenging problem that requires reasoning about the effects of a sequence of actions on a physical 3D scene. While traditional task planning methods are shown to be eff…
Object RearrangementRobot ManipulationPoint CloudsUnderstanding hand-object manipulation by modeling the contextual relationship between actions, grasp types and object attributes
This paper proposes a novel method for understanding daily hand-object manipulation by developing computer vision-based techniques. Specifically, we focus on recognizing hand grasp types, object attributes and manipulati…
ObjectGeometry-Aware Motion Latents for Learning Robust Manipulation Policies
Learning motion latents for robotic manipulation heavily relies on extracting motion patterns from visual sequences, yet effective action abstractions require understanding three-dimensional geometric transformations. He…
Point Clouds