Emphatic Temporal-Difference Learning

2015-07-06 · A. Rupam Mahmood, Huizhen Yu, Martha White, Richard S. Sutton

Emphatic algorithms are temporal-difference learning algorithms that change their effective state distribution by selectively emphasizing and de-emphasizing their updates on different time steps. Recent works by Sutton, Mahmood and White (2015), and Yu (2015) show that by varying the emphasis in a particular way, these algorithms become stable and convergent under off-policy training with linear function approximation. This paper serves as a unified summary of the available results from both works. In addition, we demonstrate the empirical benefits from the flexibility of emphatic algorithms, including state-dependent discounting, state-dependent bootstrapping, and the user-specified allocation of function approximation resources.

📄 PDF Abstract BibTeX arXiv:1507.01569

Code (0)

등록된 구현이 없습니다.

Truncated Emphatic Temporal Difference Methods for Prediction and Control

2021-08-11 · Shangtong Zhang, Shimon Whiteson

Emphatic Temporal Difference (TD) methods are a class of off-policy Reinforcement Learning (RL) methods involving the use of followon traces. Despite the theoretical success of emphatic TD methods in addressing the notor…

Prediction Reinforcement Learning (RL)

Atari Games Deep Reinforcement Learning reinforcement-learning Reinforcement Learning+1

Emphatic Temporal-Difference Learning

Code (0)

Similar Papers 제목 키워드 기반

Truncated Emphatic Temporal Difference Methods for Prediction and Control

Regularized Centered Emphatic Temporal Difference Learning

On Convergence of Emphatic Temporal-Difference Learning

Some Simulation Results for Emphatic Temporal-Difference Learning Algorithms

Emphatic Algorithms for Deep Reinforcement Learning