Maximum Entropy Semi-Supervised Inverse Reinforcement Learning
A popular approach to apprenticeship learning (AL) is to formulate it as an inverse reinforcement learning (IRL) problem. The MaxEnt-IRL algorithm successfully integrates the maximum entropy principle into IRL and unlike its predecessors, it resolves the ambiguity arising from the fact that a possibly large number of policies could match the expert's behavior. In this paper, we study an AL setting in which in addition to the expert's trajectories, a number of unsupervised trajectories is available. We introduce MESSI, a novel algorithm that combines MaxEnt-IRL with principles coming from semi-supervised learning. In particular, MESSI integrates the unsupervised data into the MaxEnt-IRL framework using a pairwise penalty on trajectories. Empirical results in a highway driving and grid-world problems indicate that MESSI is able to take advantage of the unsupervised trajectories and improve the performance of MaxEnt-IRL.
Code (0)
등록된 구현이 없습니다.
Tasks
Reinforcement LearningSimilar Papers 제목 키워드 기반
X-MEN: Guaranteed XOR-Maximum Entropy Constrained Inverse Reinforcement Learning
Inverse Reinforcement Learning (IRL) is a powerful way of learning from demonstrations. In this paper, we address IRL problems with the availability of prior knowledge that optimal policies will never violate certain con…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Generalized Maximum Causal Entropy for Inverse Reinforcement Learning
We consider the problem of learning from demonstrated trajectories with inverse reinforcement learning (IRL). Motivated by a limitation of the classical maximum entropy model in capturing the structure of the network of …
reinforcement-learningReinforcement LearningReinforcement Learning (RL)A Primer on Maximum Causal Entropy Inverse Reinforcement Learning
Inverse Reinforcement Learning (IRL) algorithms infer a reward function that explains demonstrations provided by an expert acting in the environment. Maximum Causal Entropy (MCE) IRL is currently the most popular formula…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)A proof of convergence of inverse reinforcement learning for multi-objective optimization
We show the convergence of Wasserstein inverse reinforcement learning for multi-objective optimizations with the projective subgradient method by formulating an inverse problem of the multi-objective optimization problem…
reinforcement-learningReinforcement LearningWeighted Maximum Entropy Inverse Reinforcement Learning
We study inverse reinforcement learning (IRL) and imitation learning (IM), the problems of recovering a reward or policy function from expert's demonstrated trajectories. We propose a new way to improve the learning proc…
Imitation Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)