PC-MLP: Model-based Reinforcement Learning with Policy Cover Guided Exploration

2021-07-15 · Yuda Song, Wen Sun

Model-based Reinforcement Learning (RL) is a popular learning paradigm due to its potential sample efficiency compared to model-free RL. However, existing empirical model-based RL approaches lack the ability to explore. This work studies a computationally and statistically efficient model-based algorithm for both Kernelized Nonlinear Regulators (KNR) and linear Markov Decision Processes (MDPs). For both models, our algorithm guarantees polynomial sample complexity and only uses access to a planning oracle. Experimentally, we first demonstrate the flexibility and efficacy of our algorithm on a set of exploration challenging control tasks where existing empirical model-based RL approaches completely fail. We then show that our approach retains excellent performance even in common dense reward control benchmarks that do not require heavy exploration. Finally, we demonstrate that our method can also perform reward-free exploration efficiently. Our code can be found at https://github.com/yudasong/PCMLP.

📄 PDF Abstract BibTeX arXiv:2107.07410

Code (1)

yudasong/PCMLP 공식 구현

Tasks

Model-based Reinforcement Learningreinforcement-learning Reinforcement Learning (RL)

Landmark-Guided Subgoal Generation in Hierarchical Reinforcement Learning

2021-10-26 · NeurIPS 2021 12 · Junsu Kim, Younggyo Seo, Jinwoo Shin

Goal-conditioned hierarchical reinforcement learning (HRL) has shown promising results for solving complex and long-horizon RL tasks. However, the action space of high-level policy in the goal-conditioned HRL is often la…

Efficient Exploration Hierarchical Reinforcement Learning reinforcement-learning Reinforcement Learning+1

Improving the Exploration of Deep Reinforcement Learning in Continuous Domains using Planning for Policy Search

2020-10-24 · Jakob J. Hollenstein, Erwan Renaudo, Matteo Saveriano, Justus Piater

Local policy search is performed by most Deep Reinforcement Learning (D-RL) methods, which increases the risk of getting trapped in a local minimum. Furthermore, the availability of a simulation model is not fully exploi…

Deep Reinforcement Learning Model-based Reinforcement Learning reinforcement-learning Reinforcement Learning+1

Reinforcement Learning on Web Interfaces Using Workflow-Guided Exploration

2018-02-24 · ICLR 2018 1 · Evan Zheran Liu, Kelvin Guu, Panupong Pasupat, Tianlin Shi 외

Reinforcement learning (RL) agents improve through trial-and-error, but when reward is sparse and the agent cannot discover successful action sequences, learning stagnates. This has been a notable problem in training dee…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Where-to-Learn: Analytical Policy Gradient Directed Exploration for On-Policy Robotic Reinforcement Learning

2026-03-28 · Leixin Chang, Xinchen Yao, Ben Liu, Liangjing Yang 외 arxiv

On-policy reinforcement learning (RL) algorithms have demonstrated great potential in robotic control, where effective exploration is crucial for efficient and high-quality policy learning. However, how to encourage the …

Reinforcement Learning

Optimal Control-Based Baseline for Guided Exploration in Policy Gradient Methods

2020-11-04 · Xubo Lyu, Site Li, Seth Siriya, Ye Pu 외

In this paper, a novel optimal control-based baseline function is presented for the policy gradient method in deep reinforcement learning (RL). The baseline is obtained by computing the value function of an optimal contr…

Deep Reinforcement Learning Policy Gradient Methods Reinforcement Learning (RL)