Methodical Advice Collection and Reuse in Deep Reinforcement Learning

2022-04-14 · Sahir, Ercüment İlhan, Srijita Das, Matthew E. Taylor

Reinforcement learning (RL) has shown great success in solving many challenging tasks via use of deep neural networks. Although using deep learning for RL brings immense representational power, it also causes a well-known sample-inefficiency problem. This means that the algorithms are data-hungry and require millions of training samples to converge to an adequate policy. One way to combat this issue is to use action advising in a teacher-student framework, where a knowledgeable teacher provides action advice to help the student. This work considers how to better leverage uncertainties about when a student should ask for advice and if the student can model the teacher to ask for less advice. The student could decide to ask for advice when it is uncertain or when both it and its model of the teacher are uncertain. In addition to this investigation, this paper introduces a new method to compute uncertainty for a deep RL agent using a secondary neural network. Our empirical results show that using dual uncertainties to drive advice collection and reuse may improve learning performance across several Atari games.

📄 PDF Abstract BibTeX arXiv:2204.07254

Code (0)

등록된 구현이 없습니다.

Tasks

Atari GamesDeep Reinforcement Learningreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Assessing Regulatory Risk in Personal Financial Advice Documents: a Pilot Study

2019-10-11 · Wanita Sherchan, Simon Harris, Sue Ann Chen, Nebula Alam 외

Assessing regulatory compliance of personal financial advice is currently a complex manual process. In Australia, only 5%- 15% of advice documents are audited annually and 75% of these are found to be non-compliant(ASI 2…

Uncertainty-Aware and Temporally Regulated Expert Advice in Reinforcement Learning for Autonomous Driving

2026-05-28 · Ahmed Abouelazm, Felix Klingebiel, Philip Schörner, J. Marius Zöllner arxiv

Exploration in reinforcement learning for autonomous driving is inherently unsafe: agents must experience novel behaviors to learn, yet exploration can lead to collisions or off-road driving. We propose an uncertainty-aw…

Reinforcement Learning Autonomous Driving

Action Advising with Advice Imitation in Deep Reinforcement Learning

2021-04-17 · Ercument Ilhan, Jeremy Gow, Diego Perez-Liebana

Action advising is a peer-to-peer knowledge exchange technique built on the teacher-student paradigm to alleviate the sample inefficiency problem in deep reinforcement learning. Recently proposed student-initiated approa…

Atari Games Behavioural cloning Deep Reinforcement Learning reinforcement-learning+2

A Broad-persistent Advising Approach for Deep Interactive Reinforcement Learning in Robotic Environments

2021-10-15 · Hung Son Nguyen, Francisco Cruz, Richard Dazeley

Deep Reinforcement Learning (DeepRL) methods have been widely used in robotics to learn about the environment and acquire behaviors autonomously. Deep Interactive Reinforcement Learning (DeepIRL) includes interactive fee…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+1

Accelerating Reinforcement Learning Algorithms Convergence using Pre-trained Large Language Models as Tutors With Advice Reusing

2025-09-10 · Lukas Toral, Teddy Lazebnik arxiv

Reinforcement Learning (RL) algorithms often require long training to become useful, especially in complex environments with sparse rewards. While techniques like reward shaping and curriculum learning exist to accelerat…

Reinforcement Learning