Demonstration-Guided Reinforcement Learning with Efficient Exploration for Task Automation of Surgical Robot

2023-02-20 · Tao Huang, Kai Chen, Bin Li, Yun-hui Liu, Qi Dou

Task automation of surgical robot has the potentials to improve surgical efficiency. Recent reinforcement learning (RL) based approaches provide scalable solutions to surgical automation, but typically require extensive data collection to solve a task if no prior knowledge is given. This issue is known as the exploration challenge, which can be alleviated by providing expert demonstrations to an RL agent. Yet, how to make effective use of demonstration data to improve exploration efficiency still remains an open challenge. In this work, we introduce Demonstration-guided EXploration (DEX), an efficient reinforcement learning algorithm that aims to overcome the exploration problem with expert demonstrations for surgical automation. To effectively exploit demonstrations, our method estimates expert-like behaviors with higher values to facilitate productive interactions, and adopts non-parametric regression to enable such guidance at states unobserved in demonstration data. Extensive experiments on $10$ surgical manipulation tasks from SurRoL, a comprehensive surgical simulation platform, demonstrate significant improvements in the exploration efficiency and task success rates of our method. Moreover, we also deploy the learned policies to the da Vinci Research Kit (dVRK) platform to show the effectiveness on the real robot. Code is available at https://github.com/med-air/DEX.

📄 PDF Abstract BibTeX arXiv:2302.09772

Code (2)

med-air/dex 공식 구현 pytorch

med-air/surrol tf

Tasks

Efficient Explorationreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Keyframe-Guided Structured Rewards for Reinforcement Learning in Long-Horizon Laboratory Robotics

2026-02-28 · Yibo Qiu, Shu'ang Sun, Haoliang Ye, Ronald X Xu 외 arxiv

Long-horizon precision manipulation in laboratory automation, such as pipette tip attachment and liquid transfer, requires policies that respect strict procedural logic while operating in continuous, high-dimensional sta…

Reinforcement Learning

Demonstration-Guided Continual Reinforcement Learning in Dynamic Environments

2025-12-21 · Xue Yang, Michael Schukat, Junlin Lu, Patrick Mannion 외 arxiv

Reinforcement learning (RL) excels in various applications but struggles in dynamic environments where the underlying Markov decision process evolves. Continual reinforcement learning (CRL) enables RL agents to continual…

Reinforcement Learning

Reinforcement Learning on Web Interfaces Using Workflow-Guided Exploration

2018-02-24 · ICLR 2018 1 · Evan Zheran Liu, Kelvin Guu, Panupong Pasupat, Tianlin Shi 외

Reinforcement learning (RL) agents improve through trial-and-error, but when reward is sparse and the agent cannot discover successful action sequences, learning stagnates. This has been a notable problem in training dee…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Guided Exploration with Proximal Policy Optimization using a Single Demonstration

2020-07-07 · Gabriele Libardi, Gianni de Fabritiis

Solving sparse reward tasks through exploration is one of the major challenges in deep reinforcement learning, especially in three-dimensional, partially-observable environments. Critically, the algorithm proposed in thi…

Deep Reinforcement Learning

Guided Online Distillation: Promoting Safe Reinforcement Learning by Offline Demonstration

2023-09-18 · Jinning Li, Xinyi Liu, Banghua Zhu, Jiantao Jiao 외

Safe Reinforcement Learning (RL) aims to find a policy that achieves high rewards while satisfying cost constraints. When learning from scratch, safe RL agents tend to be overly conservative, which impedes exploration an…

Autonomous Driving Decision Making reinforcement-learning Reinforcement Learning+2