Tsallis Entropy Regularization for Linearly Solvable MDP and Linear Quadratic Regulator
Shannon entropy regularization is widely adopted in optimal control due to its ability to promote exploration and enhance robustness, e.g., maximum entropy reinforcement learning known as Soft Actor-Critic. In this paper, Tsallis entropy, which is a one-parameter extension of Shannon entropy, is used for the regularization of linearly solvable MDP and linear quadratic regulators. We derive the solution for these problems and demonstrate its usefulness in balancing between exploration and sparsity of the obtained control law.
Code (0)
등록된 구현이 없습니다.
Tasks
reinforcement-learningReinforcement LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Enforcing KL Regularization in General Tsallis Entropy Reinforcement Learning via Advantage Learning
Maximum Tsallis entropy (MTE) framework in reinforcement learning has gained popularity recently by virtue of its flexible modeling choices including the widely used Shannon entropy and sparse entropy. However, non-Shann…
reinforcement-learningReinforcement Learning (RL)Tsallis Reinforcement Learning: A Unified Framework for Maximum Entropy Reinforcement Learning
In this paper, we present a new class of Markov decision processes (MDPs), called Tsallis MDPs, with Tsallis entropy maximization, which generalizes existing maximum entropy reinforcement learning (RL). A Tsallis MDP pro…
MuJoCoreinforcement-learningReinforcement LearningReinforcement Learning (RL)Continuous-time q-Learning for Jump-Diffusion Models under Tsallis Entropy
This paper studies the continuous-time reinforcement learning in jump-diffusion models by featuring the q-learning (the continuous-time counterpart of Q-learning) under Tsallis entropy regularization. Contrary to the Sha…
Q-Learning$q$-Munchausen Reinforcement Learning
The recently successful Munchausen Reinforcement Learning (M-RL) features implicit Kullback-Leibler (KL) regularization by augmenting the reward function with logarithm of the current stochastic policy. Though significan…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Nonextensive information theoretical machine
In this paper, we propose a new discriminative model named \emph{nonextensive information theoretical machine (NITM)} based on nonextensive generalization of Shannon information theory. In NITM, weight parameters are tre…