Interactively Teaching an Inverse Reinforcement Learner with Limited Feedback

2023-09-16 · Rustam Zayanov, Francisco S. Melo, Manuel Lopes

We study the problem of teaching via demonstrations in sequential decision-making tasks. In particular, we focus on the situation when the teacher has no access to the learner's model and policy, and the feedback from the learner is limited to trajectories that start from states selected by the teacher. The necessity to select the starting states and infer the learner's policy creates an opportunity for using the methods of inverse reinforcement learning and active learning by the teacher. In this work, we formalize the teaching process with limited feedback and propose an algorithm that solves this teaching problem. The algorithm uses a modified version of the active value-at-risk method to select the starting states, a modified maximum causal entropy algorithm to infer the policy, and the difficulty score ratio method to choose the teaching demonstrations. We test the algorithm in a synthetic car driving environment and conclude that the proposed algorithm is an effective solution when the learner's feedback is limited.

📄 PDF Abstract BibTeX arXiv:2309.09095

Code (1)

rzayanov/irl-teaching-limited-feedback 공식 구현

Tasks

Active Learning Decision MakingSequential Decision Making

Methods 이 논문이 사용한 방법론

Focus 설명 없음

Learner-aware Teaching: Inverse Reinforcement Learning with Preferences and Constraints

2019-06-02 · NeurIPS 2019 12 · Sebastian Tschiatschek, Ahana Ghosh, Luis Haug, Rati Devidze 외

Inverse reinforcement learning (IRL) enables an agent to learn complex behavior by observing demonstrations from a (near-)optimal policy. The typical assumption is that the learner's goal is to match the teacher's demons…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Class Teaching for Inverse Reinforcement Learners

2019-11-29 · Manuel Lopes, Francisco Melo

In this paper we propose the first machine teaching algorithm for multiple inverse reinforcement learners. Specifically, our contributions are: (i) we formally introduce the problem of teaching a sequential task to a het…

Teaching Inverse Reinforcement Learners via Features and Demonstrations

2018-10-21 · NeurIPS 2018 12 · Luis Haug, Sebastian Tschiatschek, Adish Singla

Learning near-optimal behaviour from an expert's demonstrations typically relies on the assumption that the learner knows the features that the true reward function depends on. In this paper, we study the problem of lear…

Reinforcement Learning

Interactive Teaching Algorithms for Inverse Reinforcement Learning

2019-05-28 · Parameswaran Kamalaruban, Rati Devidze, Volkan Cevher, Adish Singla

We study the problem of inverse reinforcement learning (IRL) with the added twist that the learner is assisted by a helpful teacher. More formally, we tackle the following algorithmic question: How could a teacher provid…