Accelerated Reinforcement Learning

2017-10-23 · K. Lakshmanan

Policy gradient methods are widely used in reinforcement learning algorithms to search for better policies in the parameterized policy space. They do gradient search in the policy space and are known to converge very slowly. Nesterov developed an accelerated gradient search algorithm for convex optimization problems. This has been recently extended for non-convex and also stochastic optimization. We use Nesterov's acceleration for policy gradient search in the well-known actor-critic algorithm and show the convergence using ODE method. We tested this algorithm on a scheduling problem. Here an incoming job is scheduled into one of the four queues based on the queue lengths. We see from experimental results that algorithm using Nesterov's acceleration has significantly better performance compared to algorithm which do not use acceleration. To the best of our knowledge this is the first time Nesterov's acceleration has been used with actor-critic algorithm.

📄 PDF Abstract BibTeX arXiv:1710.08070

Code (0)

등록된 구현이 없습니다.

Tasks

Policy Gradient Methodsreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)SchedulingStochastic Optimization

Accelerated Target Updates for Q-learning

2019-05-07 · Bowen Weng, Huaqing Xiong, Wei zhang

This paper studies accelerations in Q-learning algorithms. We propose an accelerated target update scheme by incorporating the historical iterates of Q functions. The idea is conceptually inspired by the momentum-based a…

Atari Games Q-Learning reinforcement-learning Reinforcement Learning+1

A Nesterov's Accelerated quasi-Newton method for Global Routing using Deep Reinforcement Learning

2020-10-15 · S. Indrapriyadarsini, Shahrzad Mahboubi, Hiroshi Ninomiya, Takeshi Kamio 외

Deep Q-learning method is one of the most popularly used deep reinforcement learning algorithms which uses deep neural networks to approximate the estimation of the action-value function. Training of the deep Q-network (…

Deep Reinforcement Learning Q-Learning reinforcement-learning Reinforcement Learning+1

Accelerated Value Iteration via Anderson Mixing

2018-09-27 · YuJun Li, Chengzhuo Ni, Guangzeng Xie, Wenhao Yang 외

Acceleration for reinforcement learning methods is an important and challenging theme. We introduce the Anderson acceleration technique into the value iteration, developing an accelerated value iteration algorithm that w…

Atari Games Q-Learning reinforcement-learning Reinforcement Learning (RL)

Mirror Descent Search and its Acceleration

2017-09-08 · Megumi Miyashita, Shiro Yano, Toshiyuki Kondo

In recent years, attention has been focused on the relationship between black-box optimiza- tion problem and reinforcement learning problem. In this research, we propose the Mirror Descent Search (MDS) algorithm which is…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Toward Safe and Accelerated Deep Reinforcement Learning for Next-Generation Wireless Networks

2022-09-16 · Ahmad M. Nagib, Hatem Abou-zeid, Hossam S. Hassanein

Deep reinforcement learning (DRL) algorithms have recently gained wide attention in the wireless networks domain. They are considered promising approaches for solving dynamic radio resource management (RRM) problems in n…

Deep Reinforcement Learning Management reinforcement-learning Reinforcement Learning (RL)+2