Papers Policy Gradient Methods
“Policy Gradient Methods” 태그가 달린 논문 382편 · 필터 해제
A unified view of entropy-regularized Markov decision processes
We propose a general framework for entropy-regularized average-reward reinforcement learning in Markov decision processes (MDPs). Our approach is based on extending the linear-programming formulation of policy optimizati…
Policy Gradient Methodsreinforcement-learningReinforcement LearningReinforcement Learning (RL)Equivalence Between Policy Gradients and Soft Q-Learning
Two of the leading approaches for model-free reinforcement learning are policy gradient methods and $Q$-learning methods. $Q$-learning methods can be effective and sample-efficient when they work, however, it is not well…
Policy Gradient MethodsQ-Learningreinforcement-learningReinforcement Learning+1Stein Variational Policy Gradient
Policy gradient methods have been successfully applied to many complex reinforcement learning problems. However, policy gradient methods suffer from high variance, slow convergence, and inefficient exploration. In this w…
Bayesian Inferencecontinuous-controlContinuous ControlPolicy Gradient Methods+3Batch Policy Gradient Methods for Improving Neural Conversation Models
We study reinforcement learning of chatbots with recurrent neural network architectures when the rewards are noisy and expensive to obtain. For instance, a chatbot used in automated customer service support can be scored…
ChatbotPolicy Gradient Methodsreinforcement-learningReinforcement Learning+1A K-fold Method for Baseline Estimation in Policy Gradient Algorithms
The high variance issue in unbiased policy-gradient methods such as VPG and REINFORCE is typically mitigated by adding a baseline. However, the baseline fitting itself suffers from the underfitting or the overfitting pro…
MuJoCoPolicy Gradient MethodsSample-efficient Deep Reinforcement Learning for Dialog Control
Representing a dialog policy as a recurrent neural network (RNN) is attractive because it handles partial observability, infers a latent representation of state, and can be optimized with supervised learning (SL) or rein…
Deep Reinforcement LearningPolicy Gradient Methodsreinforcement-learningReinforcement Learning+1Self-critical Sequence Training for Image Captioning
Recently it has been shown that policy-gradient methods for reinforcement learning can be utilized to train deep end-to-end systems directly on non-differentiable metrics for the task at hand. In this paper we consider t…
Image CaptioningPolicy Gradient Methodsreinforcement-learningReinforcement Learning+1Q-Prop: Sample-Efficient Policy Gradient with An Off-Policy Critic
Model-free deep reinforcement learning (RL) methods have been successful in a wide variety of simulated domains. However, a major obstacle facing deep RL in the real world is their high sample complexity. Batch policy gr…
continuous-controlContinuous ControlDeep Reinforcement LearningMuJoCo+4Dual Learning for Machine Translation
While neural machine translation (NMT) is making good progress in the past two years, tens of millions of bilingual sentence pairs are needed for its training. However, human labeling is very costly. To tackle this train…
Language ModelingLanguage ModellingMachine TranslationNMT+4Deep Reinforcement Learning for Dialogue Generation
Recent neural models of dialogue generation offer great promise for generating responses for conversational agents, but tend to be shortsighted, predicting utterances one at a time while ignoring their influence on futur…
ChatbotDeep Reinforcement LearningDialogue GenerationDiversity+4Policy Gradient Methods for Off-policy Control
Off-policy learning refers to the problem of learning the value function of a way of behaving, or policy, while following a different policy. Gradient-based off-policy learning algorithms, such as GTD and TDC/GQ, converg…
Policy Gradient MethodsHigh-Dimensional Continuous Control Using Generalized Advantage Estimation
Policy gradient methods are an appealing approach in reinforcement learning because they directly optimize the cumulative reward and can straightforwardly be used with nonlinear function approximators such as neural netw…
continuous-controlContinuous ControlPolicy Gradient MethodsReinforcement Learning+1Trust Region Policy Optimization
We describe an iterative procedure for optimizing policies, with guaranteed monotonic improvement. By making several approximations to the theoretically-justified procedure, we develop a practical algorithm, called Trust…
Atari GamesPolicy Gradient MethodsPolicy Gradient for Coherent Risk Measures
Several authors have recently developed risk-sensitive policy gradient methods that augment the standard expected cost minimization problem with a measure of variability in cost. These studies have focused on specific ri…
Policy Gradient MethodsReinforcement LearningEfficient Baseline-free Sampling in Parameter Exploring Policy Gradients: Super Symmetric PGPE
Policy Gradient methods that explore directly in parameter space are among the most effective and robust direct policy search methods and have drawn a lot of attention lately. The basic method from this field, Policy Gra…
Policy Gradient MethodsAdaptive Step-Size for Policy Gradient Methods
In the last decade, policy gradient methods have significantly grown in popularity in the reinforcement--learning field. In particular, they have been largely employed in motor control and robotic applications, thanks to…
Policy Gradient MethodsReinforcement LearningOptimistic policy iteration and natural actor-critic: A unifying view and a non-optimality result
Approximate dynamic programming approaches to the reinforcement learning problem are often categorized into greedy value function methods and value-based policy gradient methods. As our first main result, we show that an…
Policy Gradient MethodsReinforcement LearningAnalysis and Improvement of Policy Gradient Estimation
Policy gradient is a useful model-free reinforcement learning approach, but it tends to suffer from instability of gradient estimates. In this paper, we analyze and improve the stability of policy gradient methods. We …
Policy Gradient Methodsreinforcement-learningReinforcement LearningReinforcement Learning (RL)A reinterpretation of the policy oscillation phenomenon in approximate policy iteration
A majority of approximate dynamic programming approaches to the reinforcement learning problem can be categorized into greedy value function methods and value-based policy gradient methods. The former approach, although …
Policy Gradient MethodsReinforcement LearningOn a Connection between Importance Sampling and the Likelihood Ratio Policy Gradient
Likelihood ratio policy gradient methods have been some of the most successful reinforcement learning algorithms, especially for learning on physical systems. We describe how the likelihood ratio policy gradient can be d…
Policy Gradient MethodsReinforcement Learning