Learner-aware Teaching: Inverse Reinforcement Learning with Preferences and Constraints

2019-06-02 · NeurIPS 2019 12 · Sebastian Tschiatschek, Ahana Ghosh, Luis Haug, Rati Devidze, Adish Singla

Inverse reinforcement learning (IRL) enables an agent to learn complex behavior by observing demonstrations from a (near-)optimal policy. The typical assumption is that the learner's goal is to match the teacher's demonstrated behavior. In this paper, we consider the setting where the learner has its own preferences that it additionally takes into consideration. These preferences can for example capture behavioral biases, mismatched worldviews, or physical constraints. We study two teaching approaches: learner-agnostic teaching, where the teacher provides demonstrations from an optimal policy ignoring the learner's preferences, and learner-aware teaching, where the teacher accounts for the learner's preferences. We design learner-aware teaching algorithms and show that significant performance improvements can be achieved over learner-agnostic teaching.

📄 PDF Abstract BibTeX arXiv:1906.00429

Code (0)

등록된 구현이 없습니다.

Tasks

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Class Teaching for Inverse Reinforcement Learners

2019-11-29 · Manuel Lopes, Francisco Melo

In this paper we propose the first machine teaching algorithm for multiple inverse reinforcement learners. Specifically, our contributions are: (i) we formally introduce the problem of teaching a sequential task to a het…

Teaching Inverse Reinforcement Learners via Features and Demonstrations

2018-10-21 · NeurIPS 2018 12 · Luis Haug, Sebastian Tschiatschek, Adish Singla

Learning near-optimal behaviour from an expert's demonstrations typically relies on the assumption that the learner knows the features that the true reward function depends on. In this paper, we study the problem of lear…

Reinforcement Learning

Interactively Teaching an Inverse Reinforcement Learner with Limited Feedback

2023-09-16 · Rustam Zayanov, Francisco S. Melo, Manuel Lopes

We study the problem of teaching via demonstrations in sequential decision-making tasks. In particular, we focus on the situation when the teacher has no access to the learner's model and policy, and the feedback from th…

Active Learning Decision Making Sequential Decision Making

Interactive Teaching Algorithms for Inverse Reinforcement Learning

2019-05-28 · Parameswaran Kamalaruban, Rati Devidze, Volkan Cevher, Adish Singla

We study the problem of inverse reinforcement learning (IRL) with the added twist that the learner is assisted by a helpful teacher. More formally, we tackle the following algorithmic question: How could a teacher provid…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Iterative Machine Teaching

2017-05-30 · ICML 2017 8 · Weiyang Liu, Bo Dai, Ahmad Humayun, Charlene Tay 외

In this paper, we consider the problem of machine teaching, the inverse problem of machine learning. Different from traditional machine teaching which views the learners as batch algorithms, we study a new paradigm where…