Towards Understanding Deep Policy Gradients: A Case Study on PPO

2020-12-14 · CUHK Course IERG5350 2020 12 · Buhua Liu, CHONG YIN

Deep reinforcement learning has shown impressive performance on many decision-making problems, where deep policy gradient algorithms prevail in continuous action space tasks. Although many algorithm-level improvements on policy gradient algorithms have been proposed, recent studies have found that code-level optimizations also play a critical role in the claimed enhancement. In this paper, we further investigate several code-level optimizations for the popular Proximal Policy Optimization (PPO) algorithm, aiming to provide insights into the importance of different components in the practical implementations.\footnote{Video presentation is available at \url{https://youtu.be/M0uTLoEUwGQ}}

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Tasks

Decision MakingDeep Reinforcement Learningreinforcement-learning Reinforcement Learning (RL)

Expected Policy Gradients

2017-06-15 · Kamil Ciosek, Shimon Whiteson

We propose expected policy gradients (EPG), which unify stochastic policy gradients (SPG) and deterministic policy gradients (DPG) for reinforcement learning. Inspired by expected sarsa, EPG integrates across the action …

MuJoCo Reinforcement Learning

Implementation Matters in Deep Policy Gradients: A Case Study on PPO and TRPO

2020-05-25 · Logan Engstrom, Andrew Ilyas, Shibani Santurkar, Dimitris Tsipras 외

We study the roots of algorithmic progress in deep policy gradient algorithms through a case study on two popular algorithms: Proximal Policy Optimization (PPO) and Trust Region Policy Optimization (TRPO). Specifically, …

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning (RL)

continuous-control Continuous Control Policy Gradient Methods

Towards Understanding Deep Policy Gradients: A Case Study on PPO

Code (0)

Tasks

Similar Papers 제목 키워드 기반

Expected Policy Gradients

Implementation Matters in Deep Policy Gradients: A Case Study on PPO and TRPO

Faster Policy Learning with Continuous-Time Gradients

Fisher-Rao Gradient Flows of Linear Programs and State-Action Natural Policy Gradients

Equivalence of stochastic and deterministic policy gradients