Maximum a Posteriori Policy Optimisation
We introduce a new algorithm for reinforcement learning called Maximum aposteriori Policy Optimisation (MPO) based on coordinate ascent on a relative entropy objective. We show that several existing methods can directly be related to our derivation. We develop two off-policy algorithms and demonstrate that they are competitive with the state-of-the-art in deep reinforcement learning. In particular, for continuous control, our method outperforms existing methods with respect to sample efficiency, premature convergence and robustness to hyperparameter settings while achieving similar or better final performance.
Code (3)
Tasks
continuous-controlContinuous ControlDeep Reinforcement Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)Similar Papers 제목 키워드 기반
Probabilistic inverse reinforcement learning in unknown environments
We consider the problem of learning by demonstration from agents acting in unknown stochastic Markov environments or games. Our aim is to estimate agent preferences in order to construct improved policies for the same ta…
Bayesian Inferencereinforcement-learningReinforcement LearningReinforcement Learning (RL)Probabilistic inverse reinforcement learning in unknown environments
We consider the problem of learning by demonstration from agents acting in unknown stochastic Markov environments or games. Our aim is to estimate agent preferences in order to construct improved policies for the same ta…
Bayesian Inferencereinforcement-learningReinforcement LearningReinforcement Learning (RL)Relative Entropy Regularized Policy Iteration
We present an off-policy actor-critic algorithm for Reinforcement Learning (RL) that combines ideas from gradient-free optimization via stochastic search with learned action-value function. The result is a simple procedu…
continuous-controlContinuous ControlOpenAI GymReinforcement Learning+1Maximum Entropy On-Policy Actor-Critic via Entropy Advantage Estimation
Entropy Regularisation is a widely adopted technique that enhances policy optimisation performance and stability. A notable form of entropy regularisation is augmenting the objective with an entropy term, thereby simulta…
MuJoCoAn end-to-end data-driven optimisation framework for constrained trajectories
Many real-world problems require to optimise trajectories under constraints. Classical approaches are based on optimal control methods but require an exact knowledge of the underlying dynamics, which could be challenging…