Anderson Acceleration for Reinforcement Learning

2018-09-25 · Matthieu Geist, Bruno Scherrer

Anderson acceleration is an old and simple method for accelerating the computation of a fixed point. However, as far as we know and quite surprisingly, it has never been applied to dynamic programming or reinforcement learning. In this paper, we explain briefly what Anderson acceleration is and how it can be applied to value iteration, this being supported by preliminary experiments showing a significant speed up of convergence, that we critically discuss. We also discuss how this idea could be applied more generally to (deep) reinforcement learning.

📄 PDF Abstract BibTeX arXiv:1809.09501

Code (0)

등록된 구현이 없습니다.

Tasks

Deep Reinforcement Learningreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Methods 이 논문이 사용한 방법론

SPEED The monocular depth estimation (MDE) is the task of estimating depth from a single frame. This information is an essential knowledge in many computer vision tasks such as scene…

Damped Anderson Mixing for Deep Reinforcement Learning: Acceleration, Convergence, and Stabilization

2021-10-17 · NeurIPS 2021 12 · Ke Sun, Yafei Wang, Yi Liu, Yingnan Zhao 외

Anderson mixing has been heuristically applied to reinforcement learning (RL) algorithms for accelerating convergence and improving the sampling efficiency of deep RL. Despite its heuristic improvement of convergence, a …

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning (RL)

Accelerated Value Iteration via Anderson Mixing

2018-09-27 · YuJun Li, Chengzhuo Ni, Guangzeng Xie, Wenhao Yang 외

Acceleration for reinforcement learning methods is an important and challenging theme. We introduce the Anderson acceleration technique into the value iteration, developing an accelerated value iteration algorithm that w…

Atari Games Q-Learning reinforcement-learning Reinforcement Learning (RL)

Anderson Acceleration of Proximal Gradient Methods

2019-10-18 · ICML 2020 1 · Vien V. Mai, Mikael Johansson

Anderson acceleration is a well-established and simple technique for speeding up fixed-point computations with countless applications. Previous studies of Anderson acceleration in optimization have only been able to prov…

Regularized Anderson Acceleration for Off-Policy Deep Reinforcement Learning

2019-09-07 · NeurIPS 2019 12 · Wenjie Shi, Shiji Song, Hui Wu, Ya-Chu Hsu 외

Model-free deep reinforcement learning (RL) algorithms have been widely used for a range of complex control tasks. However, slow convergence and sample inefficiency remain challenging problems in RL, especially when hand…

Deep Reinforcement Learning MuJoCo reinforcement-learning Reinforcement Learning+1

A Fast Anderson-Chebyshev Acceleration for Nonlinear Optimization

2018-09-07 · Zhize Li, Jian Li

Anderson acceleration (or Anderson mixing) is an efficient acceleration method for fixed point iterations $x_{t+1}=G(x_t)$, e.g., gradient descent can be viewed as iteratively applying the operation $G(x) \triangleq x-\a…

subspace methods