A Bayesian Approach to Robust Inverse Reinforcement Learning
We consider a Bayesian approach to offline model-based inverse reinforcement learning (IRL). The proposed framework differs from existing offline model-based IRL approaches by performing simultaneous estimation of the expert's reward function and subjective model of environment dynamics. We make use of a class of prior distributions which parameterizes how accurate the expert's model of the environment is to develop efficient algorithms to estimate the expert's reward and subjective dynamics in high-dimensional settings. Our analysis reveals a novel insight that the estimated policy exhibits robust performance when the expert is believed (a priori) to have a highly accurate model of the environment. We verify this observation in the MuJoCo environments and show that our algorithms outperform state-of-the-art offline IRL algorithms.
Code (2)
Tasks
Imitation LearningMuJoCoreinforcement-learningReinforcement LearningSimilar Papers 제목 키워드 기반
Multi-agent Inverse Reinforcement Learning for Two-person Zero-sum Games
The focus of this paper is a Bayesian framework for solving a class of problems termed multi-agent inverse reinforcement learning (MIRL). Compared to the well-known inverse reinforcement learning (IRL) problem, MIRL is f…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Vocal Bursts Valence PredictionNecessary and Sufficient Conditions for Inverse Reinforcement Learning of Bayesian Stopping Time Problems
This paper presents an inverse reinforcement learning~(IRL) framework for Bayesian stopping time problems. By observing the actions of a Bayesian decision maker, we provide a necessary and sufficient condition to identif…
reinforcement-learningReinforcement Learning (RL)Two-sample testingNonparametric Bayesian Inverse Reinforcement Learning for Multiple Reward Functions
We present a nonparametric Bayesian approach to inverse reinforcement learning (IRL) for multiple reward functions. Most previous IRL algorithms assume that the behaviour data is obtained from an agent who is optimizing …
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Inverse Reinforcement Learning using Revealed Preferences and Passive Stochastic Optimization
This monograph, spanning three chapters, explores Inverse Reinforcement Learning (IRL). The first two chapters view inverse reinforcement learning (IRL) through the lens of revealed preferences from microeconomics while …
Stochastic OptimizationReinforcement LearningKernel Density Bayesian Inverse Reinforcement Learning
Inverse reinforcement learning (IRL) methods infer an agent's reward function using demonstrations of expert behavior. A Bayesian IRL approach models a distribution over candidate reward functions, capturing a degree of …
BIRLDensity Estimationreinforcement-learningReinforcement Learning+1