Active Learning for Risk-Sensitive Inverse Reinforcement Learning

2019-09-14 · Rui Chen, Wenshuo Wang, Zirui Zhao, Ding Zhao

One typical assumption in inverse reinforcement learning (IRL) is that human experts act to optimize the expected utility of a stochastic cost with a fixed distribution. This assumption deviates from actual human behaviors under ambiguity. Risk-sensitive inverse reinforcement learning (RS-IRL) bridges such gap by assuming that humans act according to a random cost with respect to a set of subjectively distorted distributions instead of a fixed one. Such assumption provides the additional flexibility to model human's risk preferences, represented by a risk envelope, in safe-critical tasks. However, like other learning from demonstration techniques, RS-IRL could also suffer inefficient learning due to redundant demonstrations. Inspired by the concept of active learning, this research derives a probabilistic disturbance sampling scheme to enable an RS-IRL agent to query expert support that is likely to expose unrevealed boundaries of the expert's risk envelope. Experimental results confirm that our approach accelerates the convergence of RS-IRL algorithms with lower variance while still guaranteeing unbiased convergence.

📄 PDF Abstract BibTeX arXiv:1909.07843

Code (0)

등록된 구현이 없습니다.

Tasks

Active Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Risk-Aware Active Inverse Reinforcement Learning

2019-01-08 · Daniel S. Brown, Yuchen Cui, Scott Niekum

Active learning from demonstration allows a robot to query a human for specific types of input to achieve efficient learning. Existing work has explored a variety of active query strategies; however, to our knowledge, no…

Active Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Inverse Risk-Sensitive Reinforcement Learning

2017-03-29 · Lillian J. Ratliff, Eric Mazumdar

We address the problem of inverse reinforcement learning in Markov decision processes where the agent is risk-sensitive. In particular, we model risk-sensitivity in a reinforcement learning framework by making use of mod…

Decision Making reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Bounded Risk-Sensitive Markov Games: Forward Policy Design and Inverse Reward Learning with Iterative Reasoning and Cumulative Prospect Theory

2020-09-03 · Ran Tian, Liting Sun, Masayoshi Tomizuka

Classical game-theoretic approaches for multi-agent systems in both the forward policy design problem and the inverse reward learning problem often make strong rationality assumptions: agents perfectly maximize expected …

Risk-sensitive Inverse Reinforcement Learning via Semi- and Non-Parametric Methods

2017-11-28 · Sumeet Singh, Jonathan Lacotte, Anirudha Majumdar, Marco Pavone

The literature on Inverse Reinforcement Learning (IRL) typically assumes that humans take actions in order to minimize the expected value of a cost function, i.e., that humans are risk neutral. Yet, in practice, humans a…

Decision Making reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Addressing Inherent Uncertainty: Risk-Sensitive Behavior Generation for Automated Driving using Distributional Reinforcement Learning

2021-02-05 · Julian Bernhard, Stefan Pollok, Alois Knoll

For highly automated driving above SAE level~3, behavior generation algorithms must reliably consider the inherent uncertainties of the traffic environment, e.g. arising from the variety of human driving styles. Such unc…

Distributional Reinforcement Learning Reinforcement Learning (RL)