A K-fold Method for Baseline Estimation in Policy Gradient Algorithms

2017-01-03 · Nithyanand Kota, Abhishek Mishra, Sunil Srinivasa, Xi, Chen, Pieter Abbeel

The high variance issue in unbiased policy-gradient methods such as VPG and REINFORCE is typically mitigated by adding a baseline. However, the baseline fitting itself suffers from the underfitting or the overfitting problem. In this paper, we develop a K-fold method for baseline estimation in policy gradient algorithms. The parameter K is the baseline estimation hyperparameter that can adjust the bias-variance trade-off in the baseline estimates. We demonstrate the usefulness of our approach via two state-of-the-art policy gradient algorithms on three MuJoCo locomotive control tasks.

📄 PDF Abstract BibTeX arXiv:1701.00867

Code (0)

등록된 구현이 없습니다.

Tasks

MuJoCoPolicy Gradient Methods

Deep Metric Tensor Regularized Policy Gradient

2023-05-18 · Gang Chen, Victoria Huang

Policy gradient algorithms are an important family of deep reinforcement learning techniques. Many past research endeavors focused on using the first-order policy gradient information to train policy networks. Different …

Deep Reinforcement Learning reinforcement-learning

Generalized Advantage Estimation for Distributional Policy Gradients

2025-07-23 · Shahil Shaik, Jonathon M. Smereka, Yue Wang arxiv

Generalized Advantage Estimation (GAE) has been used to mitigate the computational complexity of reinforcement learning (RL) by employing an exponentially weighted estimation of the advantage function to reduce the varia…

Reinforcement Learning OpenAI Gym

DDPG-Driven Deep-Unfolding with Adaptive Depth for Channel Estimation with Sparse Bayesian Learning

2022-01-20 · Qiyu Hu, Shuhan Shi, Yunlong Cai, Guanding Yu

Deep-unfolding neural networks (NNs) have received great attention since they achieve satisfactory performance with relatively low complexity. Typically, these deep-unfolding NNs are restricted to a fixed-depth for all i…

Smoothed functional-based gradient algorithms for off-policy reinforcement learning: A non-asymptotic viewpoint

2021-01-06 · Nithia Vijayan, Prashanth L. A

We propose two policy gradient algorithms for solving the problem of control in an off-policy reinforcement learning (RL) context. Both algorithms incorporate a smoothed functional (SF) based gradient estimation scheme. …

Off-policy evaluation Reinforcement Learning (RL)

Proximal Policy Optimization for Amortized Discrete Sampling

2026-06-14 · Anna Zykova-Myzina, Timofei Gritsaev, Daniil Tiapkin, Nikita Morozov arxiv

This paper explores policy gradient algorithms for training stochastic policies to sample from structured discrete probability distributions under the Generative Flow Network (GFlowNet) framework. Building on extensive t…

Reinforcement Learning Graph Generation