Generalizing Object-Centric Task-Axes Controllers using Keypoints
To perform manipulation tasks in the real world, robots need to operate on objects with various shapes, sizes and without access to geometric models. It is often unfeasible to train monolithic neural network policies across such large variance in object properties. Towards this generalization challenge, we propose to learn modular task policies which compose object-centric task-axes controllers. These task-axes controllers are parameterized by properties associated with underlying objects in the scene. We infer these controller parameters directly from visual input using multi-view dense correspondence learning. Our overall approach provides a simple, modular and yet powerful framework for learning manipulation tasks. We empirically evaluate our approach on multiple different manipulation tasks and show its ability to generalize to large variance in object size, shape and geometry.
Code (0)
등록된 구현이 없습니다.
Tasks
ObjectSimilar Papers 제목 키워드 기반
Learning to Compose Hierarchical Object-Centric Controllers for Robotic Manipulation
Manipulation tasks can often be decomposed into multiple subtasks performed in parallel, e.g., sliding an object to a goal pose while maintaining contact with a table. Individual subtasks can be achieved by task-axis con…
Objectreinforcement-learningReinforcement Learning (RL)Zero-shot GeneralizationGTA-2: A Multi-VLM Framework for Synthesizing Robot Manipulation Skills via Grounded Task Axes
Robotic manipulation tasks are often decomposed into behaviors or skills. However, one often needs to predefine these behaviors for specific tasks or try to cover a wide range of tasks using generic skills. As a result, …
Robot ManipulationLearning Dexterous Manipulation Policies from Experience and Imitation
We explore learning-based approaches for feedback control of a dexterous five-finger hand performing non-prehensile manipulation. First, we learn local controllers that are able to perform the task starting at a predefin…
A Geometry-Driven, Framework-Agnostic Optimization for Object Pose Estimation
Current object pose estimation research remains predominantly model-centric, focusing on architectural innovations and post-processing refinements. This paper introduces a data-centric optimization by proposing a novel, …
Pose EstimationVariational Inference for Scalable 3D Object-centric Learning
We tackle the task of scalable unsupervised object-centric representation learning on 3D scenes. Existing approaches to object-centric representation learning show limitations in generalizing to larger scenes as their le…
NeRFObjectRepresentation LearningVariational Inference