Weighted Entropy Modification for Soft Actor-Critic

2020-11-18 · Yizhou Zhao, Song-Chun Zhu

We generalize the existing principle of the maximum Shannon entropy in reinforcement learning (RL) to weighted entropy by characterizing the state-action pairs with some qualitative weights, which can be connected with prior knowledge, experience replay, and evolution process of the policy. We propose an algorithm motivated for self-balancing exploration with the introduced weight function, which leads to state-of-the-art performance on Mujoco tasks despite its simplicity in implementation.

📄 PDF Abstract BibTeX arXiv:2011.09083

Code (0)

등록된 구현이 없습니다.

Tasks

MuJoCoreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Greedy Actor-Critic: A New Conditional Cross-Entropy Method for Policy Improvement

2018-10-22 · Samuel Neumann, Sungsu Lim, Ajin Joseph, Yangchen Pan 외

Many policy gradient methods are variants of Actor-Critic (AC), where a value function (critic) is learned to facilitate updating the parameterized policy (actor). The update to the actor involves a log-likelihood update…

Policy Gradient Methods Q-Learning

Improved Soft Actor-Critic: Mixing Prioritized Off-Policy Samples with On-Policy Experience

2021-09-24 · Chayan Banerjee, Zhiyong Chen, Nasimul Noman

Soft Actor-Critic (SAC) is an off-policy actor-critic reinforcement learning algorithm, essentially based on entropy regularization. SAC trains a policy by maximizing the trade-off between expected return and entropy (ra…

continuous-control Continuous Control MuJoCo

Soft Options Critic

2019-05-23 · Elita Lobo, Scott Jordan

The option-critic architecture (Bacon, Harb, and Precup 2017) and several variants have successfully demonstrated the use of the options framework proposed by Sutton et al (Sutton, Precup, and Singh1999) to scale learnin…

Off-Policy Actor-Critic in an Ensemble: Achieving Maximum General Entropy and Effective Environment Exploration in Deep Reinforcement Learning

2019-02-14 · Gang Chen, Yiming Peng

We propose a new policy iteration theory as an important extension of soft policy iteration and Soft Actor-Critic (SAC), one of the most efficient model free algorithms for deep reinforcement learning. Supported by the n…

Deep Reinforcement Learning Reinforcement Learning

Target Entropy Annealing for Discrete Soft Actor-Critic

2021-12-06 · Yaosheng Xu, Dailin Hu, Litian Liang, Stephen Mcaleer 외

Soft Actor-Critic (SAC) is considered the state-of-the-art algorithm in continuous action space settings. It uses the maximum entropy framework for efficiency and stability, and applies a heuristic temperature Lagrange t…

Atari Games Scheduling