Papers Policy Gradient Methods
“Policy Gradient Methods” 태그가 달린 논문 382편 · 필터 해제
Policy Learning and Evaluation with Randomized Quasi-Monte Carlo
Reinforcement learning constantly deals with hard integrals, for example when computing expectations in policy evaluation and policy iteration. These integrals are rarely analytically solvable and typically estimated wit…
continuous-controlContinuous ControlPolicy Gradient MethodsReinforcement Learning (RL)Independent Policy Gradient for Large-Scale Markov Potential Games: Sharper Rates, Function Approximation, and Game-Agnostic Convergence
We examine global non-asymptotic convergence properties of policy gradient methods for multi-agent reinforcement learning (RL) problems in Markov potential games (MPG). To learn a Nash equilibrium of an MPG in which the …
Multi-agent Reinforcement LearningPolicy Gradient MethodsReinforcement Learning (RL)PAGE-PG: A Simple and Loopless Variance-Reduced Policy Gradient Method with Probabilistic Gradient Estimation
Despite their success, policy gradient methods suffer from high variance of the gradient estimate, which can result in unsatisfactory sample complexity. Recently, numerous variance-reduced extensions of policy gradient m…
Policy Gradient MethodsLeveraging class abstraction for commonsense reinforcement learning via residual policy gradient methods
Enabling reinforcement learning (RL) agents to leverage a knowledge base while learning from experience promises to advance RL in knowledge intensive domains. However, it has proven difficult to leverage knowledge that i…
Knowledge GraphsPolicy Gradient Methodsreinforcement-learningReinforcement Learning (RL)Homotopic Policy Mirror Descent: Policy Convergence, Implicit Regularization, and Improved Sample Complexity
We propose a new policy gradient method, named homotopic policy mirror descent (HPMD), for solving discounted, infinite horizon MDPs with finite state and action spaces. HPMD performs a mirror descent type policy update …
Policy Gradient MethodsUnderstanding the Effects of Second-Order Approximations in Natural Policy Gradient Reinforcement Learning
Natural policy gradient methods are popular reinforcement learning methods that improve the stability of policy gradient methods by utilizing second-order approximations to precondition the gradient with the inverse of t…
Policy Gradient Methodsreinforcement-learningReinforcement LearningReinforcement Learning (RL)On the Convergence Rates of Policy Gradient Methods
We consider infinite-horizon discounted Markov decision problems with finite state and action spaces and study the convergence rates of the projected policy gradient method and a general class of policy mirror descent me…
Policy Gradient MethodsReinforcement Learning based Sequential Batch-sampling for Bayesian Optimal Experimental Design
Engineering problems that are modeled using sophisticated mathematical methods or are characterized by expensive-to-conduct tests or experiments, are encumbered with limited budget or finite computational resources. More…
Deep Reinforcement LearningExperimental DesignPolicy Gradient Methodsreinforcement-learning+1MDPGT: Momentum-based Decentralized Policy Gradient Tracking
We propose a novel policy gradient method for multi-agent reinforcement learning, which leverages two different variance-reduction techniques and does not require large batches over iterations. Specifically, we propose a…
Multi-agent Reinforcement LearningPolicy Gradient Methodsreinforcement-learningReinforcement Learning+2Episodic Policy Gradient Training
We introduce a novel training procedure for policy gradient methods wherein episodic memory is used to optimize the hyperparameters of reinforcement learning algorithms on-the-fly. Unlike other hyperparameter searches, w…
Policy Gradient MethodsSchedulingGlobal Convergence Using Policy Gradient Methods for Model-free Markovian Jump Linear Quadratic Control
Owing to the growth of interest in Reinforcement Learning in the last few years, gradient based policy control methods have been gaining popularity for Control problems as well. And rightly so, since gradient policy meth…
Policy Gradient MethodsTime Discretization-Invariant Safe Action Repetition for Policy Gradient Methods
In reinforcement learning, continuous time is often discretized by a time scale $\delta$, to which the resulting performance is known to be highly sensitive. In this work, we seek to find a $\delta$-invariant algorithm f…
MuJoCoPolicy Gradient MethodsGlobal Optimality and Finite Sample Analysis of Softmax Off-Policy Actor Critic under State Distribution Mismatch
In this paper, we establish the global optimality and convergence rate of an off-policy actor critic algorithm in the tabular setting without using density ratio to correct the discrepancy between the state distribution …
Policy Gradient MethodsProximal Policy Optimization with Continuous Bounded Action Space via the Beta Distribution
Reinforcement learning methods for continuous control tasks have evolved in recent years generating a family of policy gradient methods that rely primarily on a Gaussian distribution for modeling a stochastic policy. How…
continuous-controlContinuous ControlOpenAI GymPolicy Gradient MethodsConvergence and Optimality of Policy Gradient Methods in Weakly Smooth Settings
Policy gradient methods have been frequently applied to problems in control and reinforcement learning with great success, yet existing convergence analysis still relies on non-intuitive, impractical and often opaque con…
Policy Gradient Methodsreinforcement-learningReinforcement Learning (RL)Beyond Exact Gradients: Convergence of Stochastic Soft-Max Policy Gradient Methods with Entropy Regularization
Entropy regularization is an efficient technique for encouraging exploration and preventing a premature convergence of (vanilla) policy gradient methods in reinforcement learning (RL). However, the theoretical understand…
Policy Gradient MethodsReinforcement Learning (RL)Local Advantage Actor-Critic for Robust Multi-Agent Deep Reinforcement Learning
Policy gradient methods have become popular in multi-agent reinforcement learning, but they suffer from high variance due to the presence of environmental stochasticity and exploring agents (i.e., non-stationarity), whic…
Deep Reinforcement LearningMulti-agent Reinforcement LearningPolicy Gradient Methodsreinforcement-learning+1Stabilizing Dynamical Systems via Policy Gradient Methods
Stabilizing an unknown control system is one of the most fundamental problems in control systems engineering. In this paper, we provide a simple, model-free algorithm for stabilizing fully observed dynamical systems. Whi…
Policy Gradient MethodsTransform2Act: Learning a Transform-and-Control Policy for Efficient Agent Design
An agent's functionality is largely determined by its design, i.e., skeletal structure and joint attributes (e.g., length, size, strength). However, finding the optimal agent design for a given function is extremely chal…
Decision MakingPolicy Gradient MethodsProgrammatic Reinforcement Learning without Oracles
Deep reinforcement learning (RL) has led to encouraging successes in many challenging control tasks. However, a deep RL model lacks interpretability due to the difficulty of identifying how the model's control logic rela…
Bilevel OptimizationDeep Reinforcement LearningPolicy Gradient Methodsreinforcement-learning+2