DINOBot: Robot Manipulation via Retrieval and Alignment with Vision Foundation Models
We propose DINOBot, a novel imitation learning framework for robot manipulation, which leverages the image-level and pixel-level capabilities of features extracted from Vision Transformers trained with DINO. When interacting with a novel object, DINOBot first uses these features to retrieve the most visually similar object experienced during human demonstrations, and then uses this object to align its end-effector with the novel object to enable effective interaction. Through a series of real-world experiments on everyday tasks, we show that exploiting both the image-level and pixel-level properties of vision foundation models enables unprecedented learning efficiency and generalisation. Videos and code are available at https://www.robot-learning.uk/dinobot.
Code (0)
등록된 구현이 없습니다.
Tasks
Imitation LearningObjectRetrievalRobot ManipulationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
RoboTok: An Internet-Scale Data Engine for Human Demonstration Retrieval and Dexterous Manipulation Learning
Robot learning increasingly depends on broad and diverse demonstrations, yet collecting robot data remains expensive and poorly suited to covering the long tail of real-world tasks. To address this bottleneck, we introdu…
On the Effectiveness of Retrieval, Alignment, and Replay in Manipulation
Imitation learning with visual observations is notoriously inefficient when addressed with end-to-end behavioural cloning methods. In this paper, we explore an alternative paradigm which decomposes reasoning into three p…
Behavioural cloningImitation LearningObjectRetrievalReGIL: Retrieval-Guided Imitation Learning from a Single Demonstration
Learning robot manipulation policies with deep neural networks from a single demonstration remains highly challenging, as even small deviations from the demonstrated trajectory can quickly compound into failure, while co…
Robot ManipulationLearning a Thousand Tasks in a Day
Humans are remarkably efficient at learning tasks from demonstrations, but today's imitation learning methods for robot manipulation often require hundreds or thousands of demonstrations per task. We investigate two fund…
Robot ManipulationQwen-RobotManip Technical Report: Alignment Unlocks Scale for Robotic Manipulation Foundation Models
Foundation models in language and multimodality achieve strong generalization by aligning heterogeneous data under a unified formulation and training at scale. In this report, we investigate whether this scaling recipe c…
Instruction Following