Accelerating Robotic Reinforcement Learning via Parameterized Action Primitives

2021-10-28 · NeurIPS 2021 12 · Murtaza Dalal, Deepak Pathak, Ruslan Salakhutdinov

Despite the potential of reinforcement learning (RL) for building general-purpose robotic systems, training RL agents to solve robotics tasks still remains challenging due to the difficulty of exploration in purely continuous action spaces. Addressing this problem is an active area of research with the majority of focus on improving RL methods via better optimization or more efficient exploration. An alternate but important component to consider improving is the interface of the RL algorithm with the robot. In this work, we manually specify a library of robot action primitives (RAPS), parameterized with arguments that are learned by an RL policy. These parameterized primitives are expressive, simple to implement, enable efficient exploration and can be transferred across robots, tasks and environments. We perform a thorough empirical study across challenging tasks in three distinct domains with image input and a sparse terminal reward. We find that our simple change to the action interface substantially improves both the learning efficiency and task performance irrespective of the underlying RL algorithm, significantly outperforming prior methods which learn skills from offline expert data. Code and videos at https://mihdalal.github.io/raps/

📄 PDF Abstract BibTeX arXiv:2110.15360

Code (0)

등록된 구현이 없습니다.

Tasks

Efficient Explorationreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Contextual Latent-Movements Off-Policy Optimization for Robotic Manipulation Skills

2020-10-26 · Samuele Tosatto, Georgia Chalvatzaki, Jan Peters

Parameterized movement primitives have been extensively used for imitation learning of robotic tasks. However, the high-dimensionality of the parameter space hinders the improvement of such primitives in the reinforcemen…

Imitation Learning Reinforcement Learning (RL)

NS-VLA: Towards Neuro-Symbolic Vision-Language-Action Models

2026-03-10 · Ziyue Zhu, Shangyang Wu, Shuai Zhao, Zhiqiu Zhao 외 arxiv

Vision-Language-Action (VLA) models are formulated to ground instructions in visual context and generate action sequences for robotic manipulation. Despite recent progress, VLA models still face challenges in learning re…

Reinforcement Learning

SAFER: Data-Efficient and Safe Reinforcement Learning Through Skill Acquisition

2021-09-29 · Dylan Z Slack, Yinlam Chow, Bo Dai, Nevan Wichers

Though many reinforcement learning (RL) problems involve learning policies in settings that are difficult to specify safety constraints and sparse rewards, current methods struggle to rapidly and safely acquire successfu…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Robotic Grasping+1

Policy Search for Motor Primitives in Robotics

2008-12-01 · NeurIPS 2008 12 · Jens Kober, Jan R. Peters

Many motor skills in humanoid robotics can be learned using parametrized motor primitives as done in imitation learning. However, most interesting motor learning problems are high-dimensional reinforcement learning probl…

Imitation Learning Policy Gradient Methods reinforcement-learning Reinforcement Learning+1

Learning Extrinsic Dexterity with Parameterized Manipulation Primitives

2023-10-26 · Shih-Min Yang, Martin Magnusson, Johannes A. Stork, Todor Stoyanov

Many practically relevant robot grasping problems feature a target object for which all grasps are occluded, e.g., by the environment. Single-shot grasp planning invariably fails in such scenarios. Instead, it is necessa…

Friction Hierarchical Reinforcement Learning Object object-detection+2