Optimistic Proximal Policy Optimization

2019-06-25 · Takahisa Imagawa, Takuya Hiraoka, Yoshimasa Tsuruoka

Reinforcement Learning, a machine learning framework for training an autonomous agent based on rewards, has shown outstanding results in various domains. However, it is known that learning a good policy is difficult in a domain where rewards are rare. We propose a method, optimistic proximal policy optimization (OPPO) to alleviate this difficulty. OPPO considers the uncertainty of the estimated total return and optimistically evaluates the policy based on that amount. We show that OPPO outperforms the existing methods in a tabular task.

📄 PDF Abstract BibTeX arXiv:1906.11075

Code (0)

등록된 구현이 없습니다.

Tasks

BIG-bench Machine Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Optimistic Distributionally Robust Policy Optimization

2020-06-14 · Jun Song, Chaoyue Zhao

Trust Region Policy Optimization (TRPO) and Proximal Policy Optimization (PPO), as the widely employed policy based reinforcement learning (RL) methods, are prone to converge to a sub-optimal solution as they limit the p…

reinforcement-learning Reinforcement Learning (RL)

A Theoretical Analysis of Optimistic Proximal Policy Optimization in Linear Markov Decision Processes

2023-05-15 · NeurIPS 2023 11

The proximal policy optimization (PPO) algorithm stands as one of the most prosperous methods in the field of reinforcement learning (RL). Despite its success, the theoretical understanding of PPO remains deficient. Spec…

2k Reinforcement Learning (RL)

Provably Efficient Exploration in Policy Optimization

2019-12-12 · ICML 2020 1 · Qi Cai, Zhuoran Yang, Chi Jin, Zhaoran Wang

While policy-based reinforcement learning (RL) achieves tremendous successes in practice, it is significantly less understood in theory, especially compared with value-based RL. In particular, it remains elusive how to d…

Efficient Exploration Reinforcement Learning Reinforcement Learning (RL)

Optimistic Policy Regularization

2026-03-06 · Mai Pham, Vikrant Vaze, Peter Chin arxiv

Deep reinforcement learning agents frequently suffer from premature convergence, where early entropy collapse causes the policy to discard exploratory behaviors before discovering globally optimal strategies. We introduc…

Reinforcement Learning Atari Games

Optimistic Natural Policy Gradient: a Simple Efficient Policy Optimization Framework for Online RL

2023-05-18 · NeurIPS 2023 11 · Qinghua Liu, Gellért Weisz, András György, Chi Jin 외

While policy optimization algorithms have played an important role in recent empirical success of Reinforcement Learning (RL), the existing theoretical understanding of policy optimization remains rather limited -- they …

Reinforcement Learning (RL)