Optimizing Neurorobot Policy under Limited Demonstration Data through Preference Regret
Robot reinforcement learning from demonstrations (RLfD) assumes that expert data is abundant; this is usually unrealistic in the real world given data scarcity as well as high collection cost. Furthermore, imitation learning algorithms assume that the data is independently and identically distributed, which ultimately results in poorer performance as gradual errors emerge and compound within test-time trajectories. We address these issues by introducing the "master your own expertise" (MYOE) framework, a self-imitation framework that enables robotic agents to learn complex behaviors from limited demonstration data samples. Inspired by human perception and action, we propose and design what we call the queryable mixture-of-preferences state space model (QMoP-SSM), which estimates the desired goal at every time step. These desired goals are used in computing the "preference regret", which is used to optimize the robot control policy. Our experiments demonstrate the robustness, adaptability, and out-of-sample performance of our agent compared to other state-of-the-art RLfD schemes. The GitHub repository that supports this work can be found at: https://github.com/rxng8/neurorobot-preference-regret-learning.
Code (0)
등록된 구현이 없습니다.
Tasks
Reinforcement LearningSimilar Papers 제목 키워드 기반
Reward Learning from Suboptimal Demonstrations with Applications in Surgical Electrocautery
Automating robotic surgery via learning from demonstration (LfD) techniques is extremely challenging. This is because surgical tasks often involve sequential decision-making processes with complex interactions of physica…
Decision MakingImitation LearningReinforcement Learning (RL)Sequential Decision MakingIntegrating Behavior Cloning and Reinforcement Learning for Improved Performance in Dense and Sparse Reward Environments
This paper investigates how to efficiently transition and update policies, trained initially with demonstrations, using off-policy actor-critic reinforcement learning. It is well-known that techniques based on Learning f…
Q-Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)Robot Policy Transfer with Online Demonstrations: An Active Reinforcement Learning Approach
Transfer Learning (TL) is a powerful tool that enables robots to transfer learned policies across different environments, tasks, or embodiments. To further facilitate this process, efforts have been made to combine it wi…
Transfer LearningA Neurorobotics Approach to Behaviour Selection based on Human Activity Recognition
Behaviour selection has been an active research topic for robotics, in particular in the field of human-robot interaction. For a robot to interact effectively and autonomously with humans, the coupling between techniques…
Activity RecognitionDecision MakingHuman Activity RecognitionD-REX: Differentiable Real-to-Sim-to-Real Engine for Learning Dexterous Grasping
Simulation provides a cost-effective and flexible platform for data generation and policy learning to develop robotic systems. However, bridging the gap between simulation and real-world dynamics remains a significant ch…