Guided Exploration with Proximal Policy Optimization using a Single Demonstration

2020-07-07 · Gabriele Libardi, Gianni de Fabritiis

Solving sparse reward tasks through exploration is one of the major challenges in deep reinforcement learning, especially in three-dimensional, partially-observable environments. Critically, the algorithm proposed in this article uses a single human demonstration to solve hard-exploration problems. We train an agent on a combination of demonstrations and own experience to solve problems with variable initial conditions. We adapt this idea and integrate it with the proximal policy optimization (PPO). The agent is able to increase its performance and to tackle harder problems by replaying its own past trajectories prioritizing them based on the obtained reward and the maximum value of the trajectory. We compare different variations of this algorithm to behavioral cloning on a set of hard-exploration tasks in the Animal-AI Olympics environment. To the best of our knowledge, learning a task in a three-dimensional environment with comparable difficulty has never been considered before using only one human demonstration.

📄 PDF Abstract BibTeX arXiv:2007.03328

Code (1)

compsciencelab/ppo_D 공식 구현 pytorch

Tasks

Deep Reinforcement Learning

Trust Region-Guided Proximal Policy Optimization

2019-01-29 · NeurIPS 2019 12 · Yuhui Wang, Hao He, Xiaoyang Tan, Yaozhong Gan

Proximal policy optimization (PPO) is one of the most popular deep reinforcement learning (RL) methods, achieving state-of-the-art performance across a wide range of challenging tasks. However, as a model-free RL method,…

Deep Reinforcement Learning Reinforcement Learning Reinforcement Learning (RL)

Deep Gaussian Process Proximal Policy Optimization

2025-11-22 · Matthijs van der Lende, Juan Cardenas-Cartagena arxiv

Uncertainty estimation for Reinforcement Learning (RL) is a critical component in control tasks where agents must balance safe exploration and efficient learning. While deep neural networks have enabled breakthroughs in …

Reinforcement Learning Continuous Control Gaussian Processes

Proximal Policy Optimization with Evolutionary Mutations

2026-01-21 · Casimir Czworkowski, Stephen Hornish, Alhassan S. Yasin arxiv

Proximal Policy Optimization (PPO) is a widely used reinforcement learning algorithm known for its stability and sample efficiency, but it often suffers from premature convergence due to limited exploration. In this pape…

Reinforcement Learning OpenAI Gym

Proximal Policy Optimization with Adaptive Exploration

2024-05-07 · Andrei Lixandru

Proximal Policy Optimization with Adaptive Exploration (axPPO) is introduced as a novel learning algorithm. This paper investigates the exploration-exploitation tradeoff within the context of reinforcement learning and a…

reinforcement-learning Reinforcement Learning

PPO-CMA: Proximal Policy Optimization with Covariance Matrix Adaptation

2018-10-05 · Perttu Hämäläinen, Amin Babadi, Xiaoxiao Ma, Jaakko Lehtinen

Proximal Policy Optimization (PPO) is a highly popular model-free reinforcement learning (RL) approach. However, we observe that in a continuous action space, PPO can prematurely shrink the exploration variance, which le…

continuous-control Continuous Control Reinforcement Learning Reinforcement Learning (RL)