ACTRCE: Augmenting Experience via Teacher’s Advice

2019-05-01 · ICLR 2019 5 · Yuhuai Wu, Harris Chan, Jamie Kiros, Sanja Fidler, Jimmy Ba

Sparse reward is one of the most challenging problems in reinforcement learning (RL). Hindsight Experience Replay (HER) attempts to address this issue by converting a failure experience to a successful one by relabeling the goals. Despite its effectiveness, HER has limited applicability because it lacks a compact and universal goal representation. We present Augmenting experienCe via TeacheR's adviCE (ACTRCE), an efficient reinforcement learning technique that extends the HER framework using natural language as the goal representation. We first analyze the differences among goal representation, and show that ACTRCE can efficiently solve difficult reinforcement learning problems in challenging 3D navigation tasks, whereas HER with non-language goal representation failed to learn. We also show that with language goal representations, the agent can generalize to unseen instructions, and even generalize to instructions with unseen lexicons. We further demonstrate it is crucial to use hindsight advice to solve challenging tasks, but we also found that little amount of hindsight advice is sufficient for the learning to take off, showing the practical aspect of the method.

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Tasks

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Methods 이 논문이 사용한 방법론

Experience Replay Experience Replay is a replay memory technique used in reinforcement learning where we store the agent’s experiences at each time-step, $e\_{t} = \left(s\_{t}, a\_{t}, r\_{t},…

ACTRCE: Augmenting Experience via Teacher's Advice For Multi-Goal Reinforcement Learning

2019-02-12 · Harris Chan, Yuhuai Wu, Jamie Kiros, Sanja Fidler 외

Sparse reward is one of the most challenging problems in reinforcement learning (RL). Hindsight Experience Replay (HER) attempts to address this issue by converting a failed experience to a successful one by relabeling t…

Multi-Goal Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Student-Initiated Action Advising via Advice Novelty

2020-10-01 · Ercument Ilhan, Jeremy Gow, Diego Perez-Liebana

Action advising is a budget-constrained knowledge exchange mechanism between teacher-student peers that can help tackle exploration and sample inefficiency problems in deep reinforcement learning (RL). Most recently, stu…

Atari Games Deep Reinforcement Learning Reinforcement Learning (RL)

Learning on a Budget via Teacher Imitation

2021-04-17 · Ercument Ilhan, Jeremy Gow, Diego Perez-Liebana

Deep Reinforcement Learning (RL) techniques can benefit greatly from leveraging prior experience, which can be either self-generated or acquired from other entities. Action advising is a framework that provides a flexibl…

Atari Games Deep Reinforcement Learning Reinforcement Learning (RL)

Theoretically-Grounded Policy Advice from Multiple Teachers in Reinforcement Learning Settings with Applications to Negative Transfer

2016-04-13 · Yusen Zhan, Haitham Bou Ammar, Matthew E. Taylor

Policy advice is a transfer learning method where a student agent is able to learn faster via advice from a teacher. However, both this and other reinforcement learning transfer methods have little theoretical analysis. …

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Transfer Learning

Methodical Advice Collection and Reuse in Deep Reinforcement Learning

2022-04-14 · Sahir, Ercüment İlhan, Srijita Das, Matthew E. Taylor

Reinforcement learning (RL) has shown great success in solving many challenging tasks via use of deep neural networks. Although using deep learning for RL brings immense representational power, it also causes a well-know…

Atari Games Deep Reinforcement Learning reinforcement-learning Reinforcement Learning+1