Incrementally Learning Functions of the Return

2019-07-05 · Brendan Bennett, Wesley Chung, Muhammad Zaheer, Vincent Liu

Temporal difference methods enable efficient estimation of value functions in reinforcement learning in an incremental fashion, and are of broader interest because they correspond learning as observed in biological systems. Standard value functions correspond to the expected value of a sum of discounted returns. While this formulation is often sufficient for many purposes, it would often be useful to be able to represent functions of the return as well. Unfortunately, most such functions cannot be estimated directly using TD methods. We propose a means of estimating functions of the return using its moments, which can be learned online using a modified TD algorithm. The moments of the return are then used as part of a Taylor expansion to approximate analytic functions of the return.

📄 PDF Abstract BibTeX arXiv:1907.04651

Code (0)

등록된 구현이 없습니다.

Tasks

Reinforcement Learning Reinforcement Learning (RL)

Universal Option Models

2014-12-01 · NeurIPS 2014 12 · Hengshuai Yao, Csaba Szepesvari, Richard S. Sutton, Joseph Modayil 외

We consider the problem of learning models of options for real-time abstract planning, in the setting where reward functions can be specified at any time and their expected returns must be efficiently computed. We introd…

Variance Penalized On-Policy and Off-Policy Actor-Critic

2021-02-03 · Arushi Jain, Gandharv Patil, Ayush Jain, Khimya Khetarpal 외

Reinforcement learning algorithms are typically geared towards optimizing the expected return of an agent. However, in many practical applications, low variance in the return is desired to ensure the reliability of an al…

MuJoCo

Lightweight image segmentation for echocardiography

2025-09-03 · Anders Kjelsrud, Lasse Løvstakken, Erik Smistad, Håvard Dalen 외 arxiv

Accurate segmentation of the left ventricle in echocardiography can enable fully automatic extraction of clinical measurements such as volumes and ejection fraction. While models configured by nnU-Net perform well, they …

Image Segmentation Data Augmentation

Intraday Functional PCA Forecasting of Cryptocurrency Returns

2025-05-26 · Joann Jasiak, Cheng Zhong

We study the Functional PCA (FPCA) forecasting method in application to functions of intraday returns on Bitcoin. We show that improved interval forecasts of future return functions are obtained when the conditional hete…

Time Series Analysis

Directly Estimating the Variance of the λ-Return Using Temporal-Difference Methods

2018-01-25 · Craig Sherstan, Brendan Bennett, Kenny Young, Dylan R. Ashley 외

This paper investigates estimating the variance of a temporal-difference learning agent's update target. Most reinforcement learning methods use an estimate of the value function, which captures how good it is for the ag…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)