Maximum Entropy Model-based Reinforcement Learning

2021-12-02 · Oleg Svidchenko, Aleksei Shpilman

Recent advances in reinforcement learning have demonstrated its ability to solve hard agent-environment interaction tasks on a super-human level. However, the application of reinforcement learning methods to practical and real-world tasks is currently limited due to most RL state-of-art algorithms' sample inefficiency, i.e., the need for a vast number of training episodes. For example, OpenAI Five algorithm that has beaten human players in Dota 2 has trained for thousands of years of game time. Several approaches exist that tackle the issue of sample inefficiency, that either offers a more efficient usage of already gathered experience or aim to gain a more relevant and diverse experience via a better exploration of an environment. However, to our knowledge, no such approach exists for model-based algorithms, that showed their high sample efficiency in solving hard control tasks with high-dimensional state space. This work connects exploration techniques and model-based reinforcement learning. We have designed a novel exploration method that takes into account features of the model-based approach. We also demonstrate through experiments that our method significantly improves the performance of the model-based algorithm Dreamer.

📄 PDF Abstract BibTeX arXiv:2112.01195

Code (0)

등록된 구현이 없습니다.

Tasks

Dota 2modelModel-based Reinforcement Learningreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Maximum Entropy Diverse Exploration: Disentangling Maximum Entropy Reinforcement Learning

2019-11-03 · Andrew Cohen, Lei Yu, Xingye Qiao, Xiangrong Tong

Two hitherto disconnected threads of research, diverse exploration (DE) and maximum entropy RL have addressed a wide range of problems facing reinforcement learning algorithms via ostensibly distinct mechanisms. In this …

Diversity reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Generalized Maximum Entropy Reinforcement Learning via Reward Shaping

2021-09-29 · Feng Tao, Yongcan Cao

Entropy regularization is a commonly used technique in reinforcement learning to improve exploration and cultivate a better pre-trained policy for later adaptation. Recent studies further show that the use of entropy reg…

MuJoCo reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Towards Simplicity in Deep Reinforcement Learning: Streamlined Off-Policy Learning

2019-09-25 · Che Wang, Yanqiu Wu, Quan Vuong, Keith Ross

The field of Deep Reinforcement Learning (DRL) has recently seen a surge in the popularity of maximum entropy reinforcement learning algorithms. Their popularity stems from the intuitive interpretation of the maximum en…

continuous-control Continuous Control Deep Reinforcement Learning MuJoCo+3

A Max-Min Entropy Framework for Reinforcement Learning

2021-06-19 · NeurIPS 2021 12 · Seungyul Han, Youngchul Sung

In this paper, we propose a max-min entropy framework for reinforcement learning (RL) to overcome the limitation of the soft actor-critic (SAC) algorithm implementing the maximum entropy RL in model-free sample-based lea…

Disentanglement reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Maximum entropy GFlowNets with soft Q-learning

2023-12-21 · Sobhan Mohammadpour, Emmanuel Bengio, Emma Frejinger, Pierre-Luc Bacon

Generative Flow Networks (GFNs) have emerged as a powerful tool for sampling discrete objects from unnormalized distributions, offering a scalable alternative to Markov Chain Monte Carlo (MCMC) methods. While GFNs draw i…

Q-Learning Reinforcement Learning (RL)