Natural Policy Gradient Methods with Parameter-based Exploration for Control Tasks
In this paper, we propose an efficient algorithm for estimating the natural policy gradient with parameter-based exploration; this algorithm samples directly in the parameter space. Unlike previous methods based on natural gradients, our algorithm calculates the natural policy gradient using the inverse of the exact Fisher information matrix. The computational cost of this algorithm is equal to that of conventional policy gradients whereas previous natural policy gradient methods have a prohibitive computational cost. Experimental results show that the proposed method outperforms several policy gradient methods.
Code (0)
등록된 구현이 없습니다.
Tasks
Policy Gradient MethodsSimilar Papers 제목 키워드 기반
Efficient Baseline-free Sampling in Parameter Exploring Policy Gradients: Super Symmetric PGPE
Policy Gradient methods that explore directly in parameter space are among the most effective and robust direct policy search methods and have drawn a lot of attention lately. The basic method from this field, Policy Gra…
Policy Gradient MethodsLast-Iterate Global Convergence of Policy Gradients for Constrained Reinforcement Learning
Constrained Reinforcement Learning (CRL) tackles sequential decision-making problems where agents are required to achieve goals by maximizing the expected return while meeting domain-specific constraints, which are often…
continuous-controlContinuous ControlDecision MakingSequential Decision MakingRethinking Policy Diversity in Ensemble Policy Gradient in Large-Scale Reinforcement Learning
Scaling reinforcement learning to tens of thousands of parallel environments requires overcoming the limited exploration capacity of a single policy. Ensemble-based policy gradient methods, which employ multiple policies…
Reinforcement LearningStein Variational Policy Gradient
Policy gradient methods have been successfully applied to many complex reinforcement learning problems. However, policy gradient methods suffer from high variance, slow convergence, and inefficient exploration. In this w…
Bayesian Inferencecontinuous-controlContinuous ControlPolicy Gradient Methods+3Fast Global Convergence of Natural Policy Gradient Methods with Entropy Regularization
Natural policy gradient (NPG) methods are among the most widely used policy optimization algorithms in contemporary reinforcement learning. This class of methods is often applied in conjunction with entropy regularizatio…
Policy Gradient Methods