Modality-Driven Design for Multi-Step Dexterous Manipulation: Insights from Neuroscience
Multi-step dexterous manipulation is a fundamental skill in household scenarios, yet remains an underexplored area in robotics. This paper proposes a modular approach, where each step of the manipulation process is addressed with dedicated policies based on effective modality input, rather than relying on a single end-to-end model. To demonstrate this, a dexterous robotic hand performs a manipulation task involving picking up and rotating a box. Guided by insights from neuroscience, the task is decomposed into three sub-skills, 1)reaching, 2)grasping and lifting, and 3)in-hand rotation, based on the dominant sensory modalities employed in the human brain. Each sub-skill is addressed using distinct methods from a practical perspective: a classical controller, a Vision-Language-Action model, and a reinforcement learning policy with force feedback, respectively. We tested the pipeline on a real robot to demonstrate the feasibility of our approach. The key contribution of this study lies in presenting a neuroscience-inspired, modality-driven methodology for multi-step dexterous manipulation.
Code (0)
등록된 구현이 없습니다.
Tasks
Vision-Language-ActionSimilar Papers 제목 키워드 기반
A Tendon-Driven Five-Fingered Hand with Distributed Tactile Perception for Dexterous Manipulation
To apply the techniques of embodied artificial intelligence to human-oid robots for complex manipulations, dexterous robotic hands are indispensable, which are restricted by the dexterity and tactile perception capabilit…
Towards Robotic Dexterous Hand Intelligence: A Survey
Robotic dexterous hands are central to contact-rich manipulation, with rapid progress driven by advances in hardware, sensing, control, simulation, and data generation. However, existing studies are often developed under…
Multi-Goal Dexterous Hand Manipulation using Probabilistic Model-based Reinforcement Learning
This paper tackles the challenge of learning multi-goal dexterous hand manipulation tasks using model-based Reinforcement Learning. We propose Goal-Conditioned Probabilistic Model Predictive Control (GC-PMPC) by designin…
Model-based Reinforcement LearningModel Predictive ControlLearning Robust Dexterous In-Hand Manipulation from Joint Sensors with Proprioceptive Transformer
In-hand object manipulation is a fundamental yet challenging capability for dexterous robots. Despite significant progress in dexterous manipulation, existing approaches rely heavily on vision or tactile sensing to track…
Reinforcement LearningMM-Hand: A 21-DOF Multi-modal Modular Dexterous Robotic Hand with Remote Actuation
High-DOF dexterous hands require compact actuation, rich sensing, and reliable thermal behavior, but conventional designs often occupy valuable in-hand space, increase end-effector mass, and suffer from heat accumulation…