A Max-Min Entropy Framework for Reinforcement Learning

2021-06-19 · NeurIPS 2021 12 · Seungyul Han, Youngchul Sung

In this paper, we propose a max-min entropy framework for reinforcement learning (RL) to overcome the limitation of the soft actor-critic (SAC) algorithm implementing the maximum entropy RL in model-free sample-based learning. Whereas the maximum entropy RL guides learning for policies to reach states with high entropy in the future, the proposed max-min entropy framework aims to learn to visit states with low entropy and maximize the entropy of these low-entropy states to promote better exploration. For general Markov decision processes (MDPs), an efficient algorithm is constructed under the proposed max-min entropy framework based on disentanglement of exploration and exploitation. Numerical results show that the proposed algorithm yields drastic performance improvement over the current state-of-the-art RL algorithms.

📄 PDF Abstract BibTeX arXiv:2106.10517

Code (1)

seungyulhan/mme 공식 구현 tf

Tasks

Disentanglement reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Generalized Maximum Entropy Reinforcement Learning via Reward Shaping

2021-09-29 · Feng Tao, Yongcan Cao

Entropy regularization is a commonly used technique in reinforcement learning to improve exploration and cultivate a better pre-trained policy for later adaptation. Recent studies further show that the use of entropy reg…

MuJoCo reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Implicit Policy for Reinforcement Learning

2018-06-10 · Yunhao Tang, Shipra Agrawal

We introduce Implicit Policy, a general class of expressive policies that can flexibly represent complex action distributions in reinforcement learning, with efficient algorithms to compute entropy regularized policy gra…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Mind Your Entropy: From Maximum Entropy to Trajectory Entropy-Constrained RL

2025-10-25 · Guojian Zhan, Likun Wang, Pengcheng Wang, Feihong Zhang 외 arxiv

Maximum entropy has become a mainstream off-policy reinforcement learning (RL) framework for balancing exploitation and exploration. However, two bottlenecks still limit further performance improvement: (1) non-stationar…

Reinforcement Learning OpenAI Gym

Tsallis Reinforcement Learning: A Unified Framework for Maximum Entropy Reinforcement Learning

2019-01-31 · Kyungjae Lee, Sungyub Kim, Sungbin Lim, Sungjoon Choi 외

In this paper, we present a new class of Markov decision processes (MDPs), called Tsallis MDPs, with Tsallis entropy maximization, which generalizes existing maximum entropy reinforcement learning (RL). A Tsallis MDP pro…

MuJoCo reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Efficient Reinforcement Learning with Semantic and Token Entropy for LLM Reasoning

2025-12-04 · Hongye Cao, Zhixin Bai, Ziyue Peng, Boyan Wang 외 arxiv

Reinforcement learning with verifiable rewards (RLVR) has demonstrated superior performance in enhancing the reasoning capability of large language models (LLMs). However, this accuracy-oriented learning paradigm often s…

Reinforcement Learning