CANDERE-COACH: Reinforcement Learning from Noisy Feedback

2024-09-23 · YuXuan Li, Srijita Das, Matthew E. Taylor

In recent times, Reinforcement learning (RL) has been widely applied to many challenging tasks. However, in order to perform well, it requires access to a good reward function which is often sparse or manually engineered with scope for error. Introducing human prior knowledge is often seen as a possible solution to the above-mentioned problem, such as imitation learning, learning from preference, and inverse reinforcement learning. Learning from feedback is another framework that enables an RL agent to learn from binary evaluative signals describing the teacher's (positive or negative) evaluation of the agent's action. However, these methods often make the assumption that evaluative teacher feedback is perfect, which is a restrictive assumption. In practice, such feedback can be noisy due to limited teacher expertise or other exacerbating factors like cognitive load, availability, distraction, etc. In this work, we propose the CANDERE-COACH algorithm, which is capable of learning from noisy feedback by a nonoptimal teacher. We propose a noise-filtering mechanism to de-noise online feedback data, thereby enabling the RL agent to successfully learn with up to 40% of the teacher feedback being incorrect. Experiments on three common domains demonstrate the effectiveness of the proposed approach.

📄 PDF Abstract BibTeX arXiv:2409.15521

Code (0)

등록된 구현이 없습니다.

Tasks

Imitation Learningreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Convergence of a Human-in-the-Loop Policy-Gradient Algorithm With Eligibility Trace Under Reward, Policy, and Advantage Feedback

2021-09-15 · Ishaan Shah, David Halpern, Kavosh Asadi, Michael L. Littman

Fluid human-agent communication is essential for the future of human-in-the-loop reinforcement learning. An agent must respond appropriately to feedback from its human trainer even before they have significant experience…

Q-Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Deep Reinforcement Learning from Policy-Dependent Human Feedback

2019-02-12 · Dilip Arumugam, Jun Ki Lee, Sophie Saskin, Michael L. Littman

To widen their accessibility and increase their utility, intelligent agents must be able to learn complex behaviors as specified by (non-expert) human users. Moreover, they will need to learn these behaviors within a rea…

Deep Reinforcement Learning Minecraft reinforcement-learning Reinforcement Learning+1

LLM-as-a-Coach: Experiential Learning for Non-Verifiable Tasks

2026-07-20 · Tianzhu Ye, Li Dong, Guanheng Chen, He Zhu 외 hf

Reinforcement learning (RL) on open-ended tasks compresses an LLM's rubric-based evaluation into a scalar reward, discarding rich textual feedback and conflating responses with distinct quality profiles. We propose Exper…

Reinforcement Learning

CRAFT: Coaching Reinforcement Learning Autonomously using Foundation Models for Multi-Robot Coordination Tasks

2025-09-17 · Seoyeon Choi, Kanghyun Ryu, Jonghoon Ock, Negar Mehr arxiv

Multi-Agent Reinforcement Learning (MARL) provides a powerful framework for learning coordination in multi-agent systems. However, applying MARL to robotics remains challenging due to their high-dimensional continuous jo…

Multi-agent Reinforcement Learning

Interactive Learning with Corrective Feedback for Policies based on Deep Neural Networks

2018-09-30 · Rodrigo Pérez-Dattari, Carlos Celemin, Javier Ruiz-del-Solar, Jens Kober

Deep Reinforcement Learning (DRL) has become a powerful strategy to solve complex decision making problems based on Deep Neural Networks (DNNs). However, it is highly data demanding, so unfeasible in physical systems for…

Car Racing Decision Making Deep Reinforcement Learning Reinforcement Learning