Papers Policy Gradient Methods
“Policy Gradient Methods” 태그가 달린 논문 382편 · 필터 해제
Continuous MDP Homomorphisms and Homomorphic Policy Gradient
Abstraction has been widely studied as a way to improve the efficiency and generalization of reinforcement learning algorithms. In this paper, we study abstraction in the continuous-control setting. We extend the definit…
continuous-controlContinuous ControlPolicy Gradient MethodsReinforcement Learning (RL)+1On the Optimization Landscape of Dynamic Output Feedback: A Case Study for Linear Quadratic Regulator
The convergence of policy gradient algorithms in reinforcement learning hinges on the optimization landscape of the underlying optimal control problem. Theoretical insights into these algorithms can often be acquired fro…
Decision MakingPolicy Gradient MethodsThe Performance Impact of Combining Agent Factorization with Different Learning Algorithms for Multiagent Coordination
Factorizing a multiagent system refers to partitioning the state- action space to individual agents and defining the interactions be- tween those agents. This so-called agent factorization is of much im- portance in r…
ManagementPolicy Gradient MethodsNatural Policy Gradients In Reinforcement Learning Explained
Traditional policy gradient methods are fundamentally flawed. Natural gradients converge quicker and better, forming the foundation of contemporary Reinforcement Learning such as Trust Region Policy Optimization (TRPO) a…
Policy Gradient Methodsreinforcement-learningReinforcement LearningReinforcement Learning (RL)Reactive Exploration to Cope with Non-Stationarity in Lifelong Reinforcement Learning
In lifelong learning, an agent learns throughout its entire life without resets, in a constantly changing environment, as we humans do. Consequently, lifelong learning comes with a plethora of research problems such as c…
Lifelong learningPolicy Gradient MethodsQ-Learningreinforcement-learning+1Towards Global Optimality in Cooperative MARL with the Transformation And Distillation Framework
Decentralized execution is one core demand in cooperative multi-agent reinforcement learning (MARL). Recently, most popular MARL algorithms have adopted decentralized policies to enable decentralized execution and use gr…
Multi-agent Reinforcement LearningPolicy Gradient Methodsreinforcement-learningReinforcement Learning (RL)Convergence and Price of Anarchy Guarantees of the Softmax Policy Gradient in Markov Potential Games
We study the performance of policy gradient methods for the subclass of Markov games known as Markov potential games (MPGs), which extends the notion of normal-form potential games to the stateful setting and includes th…
Policy Gradient MethodsHow are policy gradient methods affected by the limits of control?
We study stochastic policy gradient methods from the perspective of control-theoretic limitations. Our main result is that ill-conditioned linear systems in the sense of Doyle inevitably lead to noisy gradient estimates.…
Policy Gradient MethodsVariance Reduction for Policy-Gradient Methods via Empirical Variance Minimization
Policy-gradient methods in Reinforcement Learning(RL) are very universal and widely applied in practice but their performance suffers from the high variance of the gradient estimate. Several procedures were proposed to r…
Policy Gradient MethodsReinforcement Learning (RL)Learning Dynamics and Generalization in Reinforcement Learning
Solving a reinforcement learning (RL) problem poses two competing challenges: fitting a potentially discontinuous value function, and generalizing well to new observations. In this paper, we analyze the learning dynamics…
Policy Gradient Methodsreinforcement-learningReinforcement LearningReinforcement Learning (RL)Stochastic Second-Order Methods Improve Best-Known Sample Complexity of SGD for Gradient-Dominated Function
We study the performance of Stochastic Cubic Regularized Newton (SCRN) on a class of functions satisfying gradient dominance property with $1\le\alpha\le2$ which holds in a wide range of applications in machine learning …
Policy Gradient MethodsReinforcement Learning (RL)Second-order methodsThe Sufficiency of Off-Policyness and Soft Clipping: PPO is still Insufficient according to an Off-Policy Measure
The popular Proximal Policy Optimization (PPO) algorithm approximates the solution in a clipped policy space. Does there exist better policies outside of this space? By using a novel surrogate objective that employs the …
Efficient ExplorationPolicy Gradient MethodsMomentum-Based Policy Gradient with Second-Order Information
Variance-reduced gradient estimators for policy gradient methods have been one of the main focus of research in the reinforcement learning in recent years as they allow acceleration of the estimation process. We propose …
Policy Gradient MethodsStochastic first-order methods for average-reward Markov decision processes
We study average-reward Markov decision processes (AMDPs) and develop novel first-order methods with strong theoretical guarantees for both policy optimization and policy evaluation. Compared with intensive research effo…
Policy Gradient MethodsLearning to Constrain Policy Optimization with Virtual Trust Region
We introduce a constrained optimization method for policy gradient reinforcement learning, which uses a virtual trust region to regulate each policy update. In addition to using the proximity of one single old policy as …
Atari GamesPolicy Gradient MethodsIndependent Natural Policy Gradient Methods for Potential Games: Finite-time Global Convergence with Entropy Regularization
A major challenge in multi-agent systems is that the system complexity grows dramatically with the number of agents as well as the size of their action spaces, which is typical in real world scenarios such as autonomous …
Autonomous VehiclesPolicy Gradient MethodsSynthesis of Stabilizing Recurrent Equilibrium Network Controllers
We propose a parameterization of a nonlinear dynamic controller based on the recurrent equilibrium network, a generalization of the recurrent neural network. We derive constraints on the parameterization under which the …
Policy Gradient MethodsAsynchronous, Option-Based Multi-Agent Policy Gradient: A Conditional Reasoning Approach
Cooperative multi-agent problems often require coordination between agents, which can be achieved through a centralized policy that considers the global state. Multi-agent policy gradient (MAPG) methods are commonly used…
Hierarchical Reinforcement LearningMulti-agent Reinforcement LearningPolicy Gradient Methodsreinforcement-learning+1Non-Parametric Stochastic Policy Gradient with Strategic Retreat for Non-Stationary Environment
In modern robotics, effectively computing optimal control policies under dynamically varying environments poses substantial challenges to the off-the-shelf parametric policy gradient methods, such as the Deep Determinist…
Policy Gradient MethodsLinear convergence of a policy gradient method for some finite horizon continuous time control problems
Despite its popularity in the reinforcement learning community, a provably convergent policy gradient method for continuous space-time control problems with nonlinear state dynamics has been elusive. This paper proposes …
Policy Gradient Methodsreinforcement-learningReinforcement LearningReinforcement Learning (RL)