ACTRCE: Augmenting Experience via Teacher's Advice For Multi-Goal Reinforcement Learning

2019-02-12 · Harris Chan, Yuhuai Wu, Jamie Kiros, Sanja Fidler, Jimmy Ba

Sparse reward is one of the most challenging problems in reinforcement learning (RL). Hindsight Experience Replay (HER) attempts to address this issue by converting a failed experience to a successful one by relabeling the goals. Despite its effectiveness, HER has limited applicability because it lacks a compact and universal goal representation. We present Augmenting experienCe via TeacheR's adviCE (ACTRCE), an efficient reinforcement learning technique that extends the HER framework using natural language as the goal representation. We first analyze the differences among goal representation, and show that ACTRCE can efficiently solve difficult reinforcement learning problems in challenging 3D navigation tasks, whereas HER with non-language goal representation failed to learn. We also show that with language goal representations, the agent can generalize to unseen instructions, and even generalize to instructions with unseen lexicons. We further demonstrate it is crucial to use hindsight advice to solve challenging tasks, and even small amount of advice is sufficient for the agent to achieve good performance.

📄 PDF Abstract BibTeX arXiv:1902.04546

Code (0)

등록된 구현이 없습니다.

Tasks

Multi-Goal Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Methods 이 논문이 사용한 방법론

Experience Replay Experience Replay is a replay memory technique used in reinforcement learning where we store the agent’s experiences at each time-step, $e\_{t} = \left(s\_{t}, a\_{t}, r\_{t},…

ACTRCE: Augmenting Experience via Teacher’s Advice

2019-05-01 · ICLR 2019 5 · Yuhuai Wu, Harris Chan, Jamie Kiros, Sanja Fidler 외

Sparse reward is one of the most challenging problems in reinforcement learning (RL). Hindsight Experience Replay (HER) attempts to address this issue by converting a failure experience to a successful one by relabeling …

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Theoretically-Grounded Policy Advice from Multiple Teachers in Reinforcement Learning Settings with Applications to Negative Transfer

2016-04-13 · Yusen Zhan, Haitham Bou Ammar, Matthew E. Taylor

Policy advice is a transfer learning method where a student agent is able to learn faster via advice from a teacher. However, both this and other reinforcement learning transfer methods have little theoretical analysis. …

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Transfer Learning

Student-Initiated Action Advising via Advice Novelty

2020-10-01 · Ercument Ilhan, Jeremy Gow, Diego Perez-Liebana

Action advising is a budget-constrained knowledge exchange mechanism between teacher-student peers that can help tackle exploration and sample inefficiency problems in deep reinforcement learning (RL). Most recently, stu…

Atari Games Deep Reinforcement Learning Reinforcement Learning (RL)

Learning on a Budget via Teacher Imitation

2021-04-17 · Ercument Ilhan, Jeremy Gow, Diego Perez-Liebana

Deep Reinforcement Learning (RL) techniques can benefit greatly from leveraging prior experience, which can be either self-generated or acquired from other entities. Action advising is a framework that provides a flexibl…

Atari Games Deep Reinforcement Learning Reinforcement Learning (RL)

Explainable Action Advising for Multi-Agent Reinforcement Learning

2022-11-15 · Yue Guo, Joseph Campbell, Simon Stepputtis, Ruiyu Li 외

Action advising is a knowledge transfer technique for reinforcement learning based on the teacher-student paradigm. An expert teacher provides advice to a student during training in order to improve the student's sample …

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+1