Improving interactive reinforcement learning: What makes a good teacher?

2019-04-15 · Francisco Cruz, Sven Magg, Yukie Nagai, Stefan Wermter

Interactive reinforcement learning has become an important apprenticeship approach to speed up convergence in classic reinforcement learning problems. In this regard, a variant of interactive reinforcement learning is policy shaping which uses a parent-like trainer to propose the next action to be performed and by doing so reduces the search space by advice. On some occasions, the trainer may be another artificial agent which in turn was trained using reinforcement learning methods to afterward becoming an advisor for other learner-agents. In this work, we analyze internal representations and characteristics of artificial agents to determine which agent may outperform others to become a better trainer-agent. Using a polymath agent, as compared to a specialist agent, an advisor leads to a larger reward and faster convergence of the reward signal and also to a more stable behavior in terms of the state visit frequency of the learner-agents. Moreover, we analyze system interaction parameters in order to determine how influential they are in the apprenticeship process, where the consistency of feedback is much more relevant when dealing with different learner obedience parameters.

📄 PDF Abstract BibTeX arXiv:1904.06879

Code (0)

등록된 구현이 없습니다.

Tasks

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Methods 이 논문이 사용한 방법론

SPEED The monocular depth estimation (MDE) is the task of estimating depth from a single frame. This information is an essential knowledge in many computer vision tasks such as scene…

What Makes a Good Dataset for Knowledge Distillation?

2024-11-19 · CVPR 2025 1 · Logan Frank, Jim Davis

Knowledge distillation (KD) has been a popular and effective method for model compression. One important assumption of KD is that the teacher's original dataset will also be available when training the student. However, …

Continual Learning Knowledge Distillation Model Compression

What Makes a "Good" Data Augmentation in Knowledge Distillation -- A Statistical Perspective

2020-12-05 · Huan Wang, Suhas Lohit, Mike Jones, Yun Fu

Knowledge distillation (KD) is a general neural network training approach that uses a teacher model to guide the student model. Existing works mainly study KD from the network output side (e.g., trying to design a better…

Active Learning Data Augmentation Knowledge Distillation

Multi-trainer Interactive Reinforcement Learning System

2022-10-14 · Zhaori Guo, Timothy J. Norman, Enrico H. Gerding

Interactive reinforcement learning can effectively facilitate the agent training via human feedback. However, such methods often require the human teacher to know what is the correct action that the agent should take. In…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

What Makes a Reward Model a Good Teacher? An Optimization Perspective

2025-03-19 · Noam Razin, Zixuan Wang, Hubert Strauss, Stanley Wei 외

The success of Reinforcement Learning from Human Feedback (RLHF) critically depends on the quality of the reward model. While this quality is primarily evaluated through accuracy, it remains unclear whether accuracy full…

Language Modeling Language Modelling

Explainable Action Advising for Multi-Agent Reinforcement Learning

2022-11-15 · Yue Guo, Joseph Campbell, Simon Stepputtis, Ruiyu Li 외

Action advising is a knowledge transfer technique for reinforcement learning based on the teacher-student paradigm. An expert teacher provides advice to a student during training in order to improve the student's sample …

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+1