Cooperative Inverse Reinforcement Learning

2016-06-09 · NeurIPS 2016 12 · Dylan Hadfield-Menell, Anca Dragan, Pieter Abbeel, Stuart Russell

For an autonomous system to be helpful to humans and to pose no unwarranted risks, it needs to align its values with those of the humans in its environment in such a way that its actions contribute to the maximization of value for the humans. We propose a formal definition of the value alignment problem as cooperative inverse reinforcement learning (CIRL). A CIRL problem is a cooperative, partial-information game with two agents, human and robot; both are rewarded according to the human's reward function, but the robot does not initially know what this is. In contrast to classical IRL, where the human is assumed to act optimally in isolation, optimal CIRL solutions produce behaviors such as active teaching, active learning, and communicative actions that are more effective in achieving value alignment. We show that computing optimal joint policies in CIRL games can be reduced to solving a POMDP, prove that optimality in isolation is suboptimal in CIRL, and derive an approximate CIRL algorithm.

📄 PDF Abstract BibTeX arXiv:1606.03137

Code (2)

chanlaw/assistive-bandits tf

rgreenblatt/ai_alignment_readings

Tasks

Active Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Learning Reward Models for Cooperative Trajectory Planning with Inverse Reinforcement Learning and Monte Carlo Tree Search

2022-02-14 · Karl Kurzer, Matthias Bitzer, J. Marius Zöllner

Cooperative trajectory planning methods for automated vehicles can solve traffic scenarios that require a high degree of cooperation between traffic participants. However, for cooperative systems to integrate into human-…

Decision Making reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+1

Individual-Level Inverse Reinforcement Learning for Mean Field Games

2022-02-13 · Yang Chen, Libo Zhang, Jiamou Liu, Shuyue Hu

The recent mean field game (MFG) formalism has enabled the application of inverse reinforcement learning (IRL) methods in large-scale multi-agent systems, with the goal of inferring reward signals that can explain demons…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Data-Driven Inverse Reinforcement Learning for Expert-Learner Zero-Sum Games

2023-01-05 · Wenqian Xue, Bosen Lian, Jialu Fan, Tianyou Chai 외

In this paper, we formulate inverse reinforcement learning (IRL) as an expert-learner interaction whereby the optimal performance intent of an expert or target agent is unknown to a learner agent. The learner observes th…

reinforcement-learning Reinforcement Learning (RL)

An Efficient, Generalized Bellman Update For Cooperative Inverse Reinforcement Learning

2018-06-11 · ICML 2018 7 · Dhruv Malik, Malayandi Palaniappan, Jaime F. Fisac, Dylan Hadfield-Menell 외

Our goal is for AI systems to correctly identify and act according to their human user's objectives. Cooperative Inverse Reinforcement Learning (CIRL) formalizes this value alignment problem as a two-player game between …

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Artificial Empathy: Towards a Framework for Unsupervised Agency Detection and Policy Reconstruction

2026-08-24 · Peter Kuhn, Chris Pang, Sonakshi Chauhan arxiv

We study how an AI system can identify and model other agents in its environment from observation alone, which is a capability necessary for cooperative behaviour in the real world. This problem is less constrained than …

Reinforcement Learning