Partial Simulation for Imitation Learning
Model-based imitation learning methods require full knowledge of the transition kernel for policy evaluation. In this work, we introduce the Expert Induced Markov Decision Process (eMDP) model as a formulation of solving imitation problems using Reinforcement Learning (RL), when only partial knowledge about the transition kernel is available. The idea of eMDP is to replace the unknown transition kernel with a synthetic kernel that: a) simulate the transition of state components for which the transition kernel is known (s_r), and b) extract from demonstrations the state components for which the kernel is unknown (s_u). The next state is then stitched from the two components: s={s_r,s_u}. We describe in detail the recipe for building an eMDP and analyze the errors caused by its synthetic kernel. Our experiments include imitation tasks in multiplayer games, where the agent has to imitate one expert in the presence of other experts for whom we cannot provide a transition model. We show that combining a policy gradient algorithm with our model achieves superior performance compared to the simulation-free alternative.
Code (0)
등록된 구현이 없습니다.
Tasks
Imitation LearningReinforcement Learning (RL)Similar Papers 제목 키워드 기반
An Evolutionary Strategy based on Partial Imitation for Solving Optimization Problems
In this work we introduce an evolutionary strategy to solve combinatorial optimization tasks, i.e. problems characterized by a discrete search space. In particular, we focus on the Traveling Salesman Problem (TSP), i.e. …
Combinatorial OptimizationTraveling Salesman ProblemGuided Policy Optimization under Partial Observability
Reinforcement Learning (RL) in partially observable environments poses significant challenges due to the complexity of learning under uncertainty. While additional information, such as that available in simulations, can …
continuous-controlContinuous ControlImitation LearningReinforcement Learning (RL)Disturbance Injection under Partial Automation: Robust Imitation Learning for Long-horizon Tasks
Partial Automation (PA) with intelligent support systems has been introduced in industrial machinery and advanced automobiles to reduce the burden of long hours of human operation. Under PA, operators perform manual oper…
Imitation LearningMixture of partially linear experts
In the mixture of experts model, a common assumption is the linearity between a response variable and covariates. While this assumption has theoretical and computational benefits, it may lead to suboptimal estimates by o…
Mixture-of-ExpertsSimulation Studies on Deep Reinforcement Learning for Building Control with Human Interaction
The building sector consumes the largest energy in the world, and there have been considerable research interests in energy consumption and comfort management of buildings. Inspired by recent advances in reinforcement le…
Deep Reinforcement LearningManagementreinforcement-learningReinforcement Learning (RL)