A Hierarchical Bayesian model for Inverse RL in Partially-Controlled Environments
Robots learning from observations in the real world using inverse reinforcement learning (IRL) may encounter objects or agents in the environment, other than the expert, that cause nuisance observations during the demonstration. These confounding elements are typically removed in fully-controlled environments such as virtual simulations or lab settings. When complete removal is impossible the nuisance observations must be filtered out. However, identifying the source of observations when large amounts of observations are made is difficult. To address this, we present a hierarchical Bayesian model that incorporates both the expert's and the confounding elements' observations thereby explicitly modeling the diverse observations a robot may receive. We extend an existing IRL algorithm originally designed to work under partial occlusion of the expert to consider the diverse observations. In a simulated robotic sorting domain containing both occlusion and confounding elements, we demonstrate the model's effectiveness. In particular, our technique outperforms several other comparative methods, second only to having perfect knowledge of the subject's trajectory.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Exploring Hierarchy-Aware Inverse Reinforcement Learning
We introduce a new generative model for human planning under the Bayesian Inverse Reinforcement Learning (BIRL) framework which takes into account the fact that humans often plan using hierarchical strategies. We describ…
BIRLreinforcement-learningReinforcement LearningReinforcement Learning (RL)Optimization-Based MCMC Methods for Nonlinear Hierarchical Statistical Inverse Problems
In many hierarchical inverse problems, not only do we want to estimate high- or infinite-dimensional model parameters in the parameter-to-observable maps, but we also have to estimate hyperparameters that represent criti…
Necessary and Sufficient Conditions for Inverse Reinforcement Learning of Bayesian Stopping Time Problems
This paper presents an inverse reinforcement learning~(IRL) framework for Bayesian stopping time problems. By observing the actions of a Bayesian decision maker, we provide a necessary and sufficient condition to identif…
reinforcement-learningReinforcement Learning (RL)Two-sample testingQuantum Bayesian Networks Can Speed up Reinforcement Learning in Partially Observable Environments
Reinforcement learning (RL) provides a principled framework for decision-making in partially observable environments, which can be modeled as Markov decision processes and compactly represented through dynamic decision B…
Reinforcement LearningBayesian hierarchical stacking: Some models are (somewhere) useful
Stacking is a widely used model averaging technique that asymptotically yields optimal predictions among linear averages. We show that stacking is most effective when model predictive performance is heterogeneous in inpu…
Bayesian InferenceTime SeriesTime Series Analysis