Distributed Reinforcement Learning of Targeted Grasping with Active Vision for Mobile Manipulators
Developing personal robots that can perform a diverse range of manipulation tasks in unstructured environments necessitates solving several challenges for robotic grasping systems. We take a step towards this broader goal by presenting the first RL-based system, to our knowledge, for a mobile manipulator that can (a) achieve targeted grasping generalizing to unseen target objects, (b) learn complex grasping strategies for cluttered scenes with occluded objects, and (c) perform active vision through its movable wrist camera to better locate objects. The system is informed of the desired target object in the form of a single, arbitrary-pose RGB image of that object, enabling the system to generalize to unseen objects without retraining. To achieve such a system, we combine several advances in deep reinforcement learning and present a large-scale distributed training system using synchronous SGD that seamlessly scales to multi-node, multi-GPU infrastructure to make rapid prototyping easier. We train and evaluate our system in a simulated environment, identify key components for improving performance, analyze its behaviors, and transfer to a real-world setup.
Code (0)
등록된 구현이 없습니다.
Tasks
Deep Reinforcement LearningGPUreinforcement-learningReinforcement Learning (RL)Robotic GraspingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Efficient Representations of Object Geometry for Reinforcement Learning of Interactive Grasping Policies
Grasping objects of different shapes and sizes - a foundational, effortless skill for humans - remains a challenging task in robotics. Although model-based approaches can predict stable grasp configurations for known obj…
Objectreinforcement-learningReinforcement Learning (RL)Transferable Active Grasping and Real Embodied Dataset
Grasping in cluttered scenes is challenging for robot vision systems, as detection accuracy can be hindered by partial occlusion of objects. We adopt a reinforcement learning (RL) framework and 3D vision architectures to…
Reinforcement LearningReinforcement Learning (RL)Active Perception and Representation for Robotic Manipulation
The vast majority of visual animals actively control their eyes, heads, and/or bodies to direct their gaze toward different parts of their environment. In contrast, recent applications of reinforcement learning in roboti…
Q-LearningReinforcement LearningRepresentation LearningVision-based interface for grasping intention detection and grip selection : towards intuitive upper-limb assistive devices
Assistive devices for indivuals with upper-limb movement often lack controllability and intuitiveness, in particular for grasping function. In this work, we introduce a novel user interface for grasping movement control …
VISO-Grasp: Vision-Language Informed Spatial Object-centric 6-DoF Active View Planning and Grasping in Clutter and Invisibility
We propose VISO-Grasp, a novel vision-language-informed system designed to systematically address visibility constraints for grasping in severely occluded environments. By leveraging Foundation Models (FMs) for spatial r…
Spatial Reasoning