Sample-efficient reinforcement learning using deep Gaussian processes

2020-11-02 · Charles Gadd, Markus Heinonen, Harri Lähdesmäki, Samuel Kaski

Reinforcement learning provides a framework for learning to control which actions to take towards completing a task through trial-and-error. In many applications observing interactions is costly, necessitating sample-efficient learning. In model-based reinforcement learning efficiency is improved by learning to simulate the world dynamics. The challenge is that model inaccuracies rapidly accumulate over planned trajectories. We introduce deep Gaussian processes where the depth of the compositions introduces model complexity while incorporating prior knowledge on the dynamics brings smoothness and structure. Our approach is able to sample a Bayesian posterior over trajectories. We demonstrate highly improved early sample-efficiency over competing methods. This is shown across a number of continuous control tasks, including the half-cheetah whose contact dynamics have previously posed an insurmountable problem for earlier sample-efficient Gaussian process based models.

📄 PDF Abstract BibTeX arXiv:2011.01226

Code (0)

등록된 구현이 없습니다.

Tasks

continuous-control Continuous Control Gaussian ProcessesModel-based Reinforcement Learningreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Methods 이 논문이 사용한 방법론

Gaussian Process Gaussian Processes are non-parametric models for approximating functions. They rely upon a measure of similarity between points (the kernel function) to predict the value for…

Autoregressive Policies for Continuous Control Deep Reinforcement Learning

2019-03-27 · Dmytro Korenkevych, A. Rupam Mahmood, Gautham Vasan, James Bergstra

Reinforcement learning algorithms rely on exploration to discover new behaviors, which is typically achieved by following a stochastic policy. In continuous control tasks, policies with a Gaussian distribution have been …

continuous-control Continuous Control Deep Reinforcement Learning reinforcement-learning+2

Upgrading from Gaussian Processes to Student's-T Processes

2018-01-18 · Brendan D. Tracey, David H. Wolpert

Gaussian process priors are commonly used in aerospace design for performing Bayesian optimization. Nonetheless, Gaussian processes suffer two significant drawbacks: outliers are a priori assumed unlikely, and the poster…

Bayesian Optimization Gaussian Processes

Pathwise Conditioning of Gaussian Processes

2020-11-08 · James T. Wilson, Viacheslav Borovitskiy, Alexander Terenin, Peter Mostowsky 외

As Gaussian processes are used to answer increasingly complex questions, analytic solutions become scarcer and scarcer. Monte Carlo methods act as a convenient bridge for connecting intractable mathematical expressions w…

Gaussian Processes global-optimization

Posterior Variance Analysis of Gaussian Processes with Application to Average Learning Curves

2019-06-04 · Armin Lederer, Jonas Umlauft, Sandra Hirche

The posterior variance of Gaussian processes is a valuable measure of the learning error which is exploited in various applications such as safe reinforcement learning and control design. However, suitable analysis of th…

Gaussian Processes reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+1

Distributionally Robust Model-based Reinforcement Learning with Large State Spaces

2023-09-05 · Shyam Sundhar Ramesh, Pier Giuseppe Sessa, Yifan Hu, Andreas Krause 외

Three major challenges in reinforcement learning are the complex dynamical systems with large state spaces, the costly data acquisition processes, and the deviation of real-world dynamics from the training environment de…

Gaussian Processes Model-based Reinforcement Learning reinforcement-learning Reinforcement Learning