Papers Policy Gradient Methods
“Policy Gradient Methods” 태그가 달린 논문 382편 · 필터 해제
Distributional constrained reinforcement learning for supply chain optimization
This work studies reinforcement learning (RL) in the context of multi-period supply chains subject to constraints, e.g., on production and inventory. We introduce Distributional Constrained Policy Optimization (DCPO), a …
Distributional Reinforcement LearningPolicy Gradient Methodsreinforcement-learningReinforcement Learning+1Accelerating Policy Gradient by Estimating Value Function from Prior Computation in Deep Reinforcement Learning
This paper investigates the use of prior computation to estimate the value function to improve sample efficiency in on-policy policy gradient methods in reinforcement learning. Our approach is to estimate the value funct…
Deep Reinforcement LearningPolicy Gradient MethodsPolicy Gradient for Rectangular Robust Markov Decision Processes
Policy gradient methods have become a standard for training reinforcement learning agents in a scalable and efficient manner. However, they do not account for transition uncertainty, whereas learning robust policies can …
FormPolicy Gradient MethodsSoftTreeMax: Exponential Variance Reduction in Policy Gradient via Tree Search
Despite the popularity of policy gradient methods, they are known to suffer from large variance and high sample complexity. To mitigate this, we introduce SoftTreeMax -- a generalization of softmax that takes planning in…
GPUPolicy Gradient MethodsStochastic Dimension-reduced Second-order Methods for Policy Optimization
In this paper, we propose several new stochastic second-order algorithms for policy optimization that only require gradient and Hessian-vector product in each iteration, making them computationally efficient and comparab…
Policy Gradient MethodsSecond-order methodsPartial advantage estimator for proximal policy optimization
Estimation of value in policy gradient methods is a fundamental problem. Generalized Advantage Estimation (GAE) is an exponentially-weighted estimator of an advantage function similar to $\lambda$-return. It substantiall…
MuJoCoPolicy Gradient MethodsOn the Global Convergence of Risk-Averse Policy Gradient Methods with Expected Conditional Risk Measures
Risk-sensitive reinforcement learning (RL) has become a popular tool for controlling the risk of uncertain outcomes and ensuring reliable performance in highly stochastic sequential decision-making problems. While Policy…
Decision MakingPolicy Gradient MethodsReinforcement Learning (RL)Sequential Decision MakingOn the Convergence of Discounted Policy Gradient Methods
Many popular policy gradient methods for reinforcement learning follow a biased approximation of the policy gradient known as the discounted approximation. While it has been shown that the discounted approximation of the…
Policy Gradient Methodsreinforcement-learningReinforcement Learning (RL)Improving a sequence-to-sequence nlp model using a reinforcement learning policy algorithm
Nowadays, the current neural network models of dialogue generation(chatbots) show great promise for generating answers for chatty agents. But they are short-sighted in that they predict utterances one at a time while dis…
ChatbotDeep Reinforcement LearningDialogue GenerationDiversity+4Policy Gradient in Robust MDPs with Global Convergence Guarantee
Robust Markov decision processes (RMDPs) provide a promising framework for computing reliable policies in the face of model errors. Many successful reinforcement learning algorithms build on variations of policy-gradient…
Policy Gradient MethodsAn Improved Analysis of (Variance-Reduced) Policy Gradient and Natural Policy Gradient Methods
In this paper, we revisit and improve the convergence of policy gradient (PG), natural PG (NPG) methods, and their variance-reduced variants, under general smooth policy parametrizations. More specifically, with the Fish…
Policy Gradient MethodsGeometry and convergence of natural policy gradient methods
We study the convergence of several natural policy gradient (NPG) methods in infinite-horizon discounted Markov decision processes with regular policy parametrizations. For a variety of NPGs and reward functions we show …
Policy Gradient MethodsConvergence of policy gradient methods for finite-horizon exploratory linear-quadratic control problems
We study the global linear convergence of policy gradient (PG) methods for finite-horizon continuous-time exploratory linear-quadratic control (LQC) problems. The setting includes stochastic LQC problems with indefinite …
Policy Gradient MethodsSymmetric (Optimistic) Natural Policy Gradient for Multi-agent Learning with Parameter Convergence
Multi-agent interactions are increasingly important in the context of reinforcement learning, and the theoretical foundations of policy gradient methods have attracted surging research interest. We investigate the global…
Policy Gradient MethodsPolicy Gradient Methods for Designing Dynamic Output Feedback Controllers
This paper proposes model-based and model-free policy gradient methods (PGMs) for designing dynamic output feedback controllers for discrete-time partially observable systems. To fulfill this objective, we first show tha…
Policy Gradient MethodsOn the convergence of policy gradient methods to Nash equilibria in general stochastic games
Learning in stochastic games is a notoriously difficult problem because, in addition to each other's strategic decisions, the players must also contend with the fact that the game itself evolves over time, possibly in a …
Policy Gradient MethodsLinear Convergence of Natural Policy Gradient Methods with Log-Linear Policies
We consider infinite-horizon discounted Markov decision processes and study the convergence rates of the natural policy gradient (NPG) and the Q-NPG methods with the log-linear policy class. Using the compatible function…
Policy Gradient MethodsIs Reinforcement Learning (Not) for Natural Language Processing: Benchmarks, Baselines, and Building Blocks for Natural Language Policy Optimization
We tackle the problem of aligning pre-trained large language models (LMs) with human preferences. If we view text generation as a sequential decision-making problem, reinforcement learning (RL) appears to be a natural co…
Decision MakingPolicy Gradient Methodsreinforcement-learningReinforcement Learning (RL)+2SoftTreeMax: Policy Gradient with Tree Search
Policy-gradient methods are widely used for learning control policies. They can be easily distributed to multiple workers and reach state-of-the-art results in many domains. Unfortunately, they exhibit large variance and…
Policy Gradient MethodsAsynchronous Actor-Critic for Multi-Agent Reinforcement Learning
Synchronizing decisions across multiple agents in realistic settings is problematic since it requires agents to wait for other agents to terminate and communicate about termination reliably. Ideally, agents should learn …
Decision MakingMulti-agent Reinforcement LearningPolicy Gradient Methodsreinforcement-learning+2