Primitive Skill-based Robot Learning from Human Evaluative Feedback

2023-07-28 · Ayano Hiranaka, Minjune Hwang, Sharon Lee, Chen Wang, Li Fei-Fei, Jiajun Wu, Ruohan Zhang

Reinforcement learning (RL) algorithms face significant challenges when dealing with long-horizon robot manipulation tasks in real-world environments due to sample inefficiency and safety issues. To overcome these challenges, we propose a novel framework, SEED, which leverages two approaches: reinforcement learning from human feedback (RLHF) and primitive skill-based reinforcement learning. Both approaches are particularly effective in addressing sparse reward issues and the complexities involved in long-horizon tasks. By combining them, SEED reduces the human effort required in RLHF and increases safety in training robot manipulation with RL in real-world settings. Additionally, parameterized skills provide a clear view of the agent's high-level intentions, allowing humans to evaluate skill choices before they are executed. This feature makes the training process even safer and more efficient. To evaluate the performance of SEED, we conducted extensive experiments on five manipulation tasks with varying levels of complexity. Our results show that SEED significantly outperforms state-of-the-art RL algorithms in sample efficiency and safety. In addition, SEED also exhibits a substantial reduction of human effort compared to other RLHF methods. Further details and video results can be found at https://seediros23.github.io/.

📄 PDF Abstract BibTeX arXiv:2307.15801

Code (0)

등록된 구현이 없습니다.

Tasks

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Robot Manipulation

From Local Corrections to Generalized Skills: Improving Neuro-Symbolic Policies with MEMO

2026-03-04 · Benjamin A. Christie, Yinlong Dai, Mohammad Bararjanianbahnamiri, Simon Stepputtis 외 arxiv

Recent works use a neuro-symbolic framework for general manipulation policies. The advantage of this framework is that -- by applying off-the-shelf vision and language models -- the robot can break complex tasks down int…

Neural Associative Skill Memories for safer robotics and modelling human sensorimotor repertoires

2025-05-14 · Pranav Mahajan, Mufeng Tang, T. Ed Li, Ioannis Havoutis 외

Modern robots face challenges shared by humans, where machines must learn multiple sensorimotor skills and express them adaptively. Equipping robots with a human-like memory of how it feels to do multiple stereotypical m…

Fault Detection

GTA-2: A Multi-VLM Framework for Synthesizing Robot Manipulation Skills via Grounded Task Axes

2026-09-09 · M. Yunus Seker, Shobhit Aggarwal, Ruwan Wickramarachchi, Jonathan Francis 외 arxiv

Robotic manipulation tasks are often decomposed into behaviors or skills. However, one often needs to predefine these behaviors for specific tasks or try to cover a wide range of tasks using generic skills. As a result, …

Robot Manipulation

Growing with Your Embodied Agent: A Human-in-the-Loop Lifelong Code Generation Framework for Long-Horizon Manipulation Skills

2025-09-23 · Yuan Meng, Zhenguo Sun, Max Fest, Xukun Li 외 arxiv

Large language models (LLMs)-based code generation for robotic manipulation has recently shown promise by directly translating human instructions into executable code, but existing methods remain noisy, constrained by fi…

Code Generation

GAN-Based Interactive Reinforcement Learning from Demonstration and Human Evaluative Feedback

2021-04-14 · Jie Huang, Rongshun Juan, Randy Gomez, Keisuke Nakamura 외

Deep reinforcement learning (DRL) has achieved great successes in many simulated tasks. The sample inefficiency problem makes applying traditional DRL methods to real-world robots a great challenge. Generative Adversaria…

Deep Reinforcement Learning Imitation Learning reinforcement-learning Reinforcement Learning+1