Wasserstein Robust Reinforcement Learning
Reinforcement learning algorithms, though successful, tend to over-fit to training environments hampering their application to the real-world. This paper proposes $\text{W}\text{R}^{2}\text{L}$ -- a robust reinforcement learning algorithm with significant robust performance on low and high-dimensional control tasks. Our method formalises robust reinforcement learning as a novel min-max game with a Wasserstein constraint for a correct and convergent solver. Apart from the formulation, we also propose an efficient and scalable solver following a novel zero-order optimisation method that we believe can be useful to numerical optimisation in general. We empirically demonstrate significant gains compared to standard and robust state-of-the-art algorithms on high-dimensional MuJuCo environments.
Code (0)
등록된 구현이 없습니다.
Tasks
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Similar Papers 제목 키워드 기반
A proof of imitation of Wasserstein inverse reinforcement learning for multi-objective optimization
We prove Wasserstein inverse reinforcement learning enables the learner's reward values to imitate the expert's reward values in a finite iteration for multi-objective optimizations. Moreover, we prove Wasserstein invers…
reinforcement-learningReinforcement LearningReinforcement Learning with Wasserstein Distance Regularisation, with Applications to Multipolicy Learning
We describe an application of Wasserstein distance to Reinforcement Learning. The Wasserstein distance in question is between the distribution of mappings of trajectories of a policy into some metric space, and some othe…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Visual Transfer for Reinforcement Learning via Wasserstein Domain Confusion
We introduce Wasserstein Adversarial Proximal Policy Optimization (WAPPO), a novel algorithm for visual transfer in Reinforcement Learning that explicitly learns to align the distributions of extracted features between a…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Wasserstein Adaptive Value Estimation for Actor-Critic Reinforcement Learning
We present Wasserstein Adaptive Value Estimation for Actor-Critic (WAVE), an approach to enhance stability in deep reinforcement learning through adaptive Wasserstein regularization. Our method addresses the inherent ins…
Computational EfficiencyDeep Reinforcement Learningreinforcement-learningReinforcement LearningEfficient Wasserstein Natural Gradients for Reinforcement Learning
A novel optimization approach is proposed for application to policy gradient methods and evolution strategies for reinforcement learning (RL). The procedure uses a computationally efficient Wasserstein natural gradient (…
Policy Gradient Methodsreinforcement-learningReinforcement LearningReinforcement Learning (RL)